Search

Industrial collaborative robots play an essential role in smart manufacturing because they improve productivity while also ensuring workplace safety. However, the development of prognostic and health management systems to ensure the reliability of these robots has been a major challenge due to the lack of fault data. This paper proposed a digital twin scheme based on the fusion of the robot kinematic and dynamic models' information down to the powertrains (i.e., the joints motor, and gear) along with the control algorithms and uncertainty accommodation based upon deep learning. The presented digital twin concept has the potential to propel simulation-based fault prediction. We also highlight and discuss challenges and opportunities around the development of the hybrid digital twin for condition monitoring of industrial collaborative robots. [ABSTRACT FROM AUTHOR]

The absence of effective quality prediction methods for functional composite materials (FCMs) produced by laser additive remanufacturing (LARM) hampers their application due to the complex cross-scale defects, including surface cracks and thermal damage to the internal reinforcement phase. This paper presents a multi-task deep learning-empowered digital twin for predicting visible and invisible defects in the fabricating process of FCMs. The dimensions of FCM trajectory, thermal damage to the reinforcement phase, and forming cracks were predicted via a parallel multi-task deep learning model. The dynamic visualization of the digital twin is realized through cross-sectional modeling and provides an intuitive and effective perception for monitoring the process. [ABSTRACT FROM AUTHOR]

Production sequencing methodology using DeepLearning Seq2Seq-LSTM is applied to a tire production case study in Quebec, Canada. Production and demand data are used to predict the most likely product sequences to operate. The comparison of 4 forecasting models, differing in consideration of demand and a statistical component, leads to nearly 70% of good prediction when all machines are studied and 10 production scenarios are considered. This performance reaches 92% for a specific class of machines. The analysis of the forecasts by class of machine allows highlighting 2 factors influencing the performance of the models, namely the ratio of product/machine by class and the total number of available records. The forecasts of possible production scenarios can then be used in a digital twin to evaluate a reasonable number of options and develop a decision support system for production sequencing. [ABSTRACT FROM AUTHOR]

The thermal errors in ball screws present complex spatial-temporal characteristics and exhibit persistent long-term memory, significantly affecting the machining accuracy of the whole machine tool in applications. Deep learning has emerged as a promising approach to predict these errors using thermal data. However, the scarcity of extensive thermal datasets, typically due to the prohibitive costs and time required for experimental acquisition, hampers the comprehensive exploitation of spatial-temporal and long-term memory attributes. Moreover, the current thermal error compensation system does not support the deployment of deep learning-based thermal error models due to its weak real-time performance. These limitations lead to suboptimal prediction accuracy and inadequate error compensation. Addressing this challenge, a novel spatial-temporal interactive integration network is proposed, and this is a sophisticated model that synergistically blends a time memory gate and a spatial-temporal fusion gate within an advanced attention-based spatial-temporal graph convolutional framework. Our network ingeniously leverages spatial data to anchor long-term memory while employing temporal data for selective memory feature extraction, facilitating a refined integration of spatial-temporal features. Concurrently, the integration of gated recurrent units and a time attention layer meticulously extracts and enhances temporal features, bolstered by our innovative time memory gate, which is adept at handling small-sample scenarios and refining thermal error predictions. Critically, the digital twin system for thermal error compensation is constructed to improve the system's real-time performance. The integration of this network into the digital twin system marks a pivotal advancement in thermal error compensation. Our empirical results underscore the system's remarkable robustness and superior predictive accuracy, demonstrating a significant reduction in positioning and machining errors (over 90% and 80% respectively) even with constrained thermal data inputs. These advancements not only delineate a substantial leap in predictive accuracy but also underscore our contribution to reducing operational costs and enhancing the efficacy of machine tools. [Display omitted] • Spatial-temporal fusion gate is proposed to extract spatiotemporal features. • STIIN is proposed to extract spatiotemporal features of thermal error. • Digital twin system for thermal error compensation is established. • STIIN model is embedded into digital twin system. • Robust compensation is realized with small-volume thermal information data. [ABSTRACT FROM AUTHOR]

The concept of digital twins, virtual representations of the physical environment that incorporate data and evolve over time, has gained momentum in the geospatial industry. One promising application of digital twins is in city planning and management, with revenue generated from digital twins expected to rise from $331 million in 2022 to $2.5 billion by 2031. Building information modeling (BIM) plays an integral role in creating accurate digital twins for industrial facilities and the urban environment. The implementation of a digital twin is highlighted in the goal of a smart city, which aims to use digitally managed infrastructure to achieve a more sustainable and livable municipality. The integration and management of municipal data, including the creation of 3D models, are key to the success of digital twins in cities. The return on investment for digital twins primarily comes from facility operation and management after construction is completed. Correct implementation of digital twins requires a significant investment in infrastructure, data governance, coordination, and collaboration. However, the drop in the cost of IT infrastructure and the ability to share costs across departments contribute to the affordability of digital twins. A mature digital twin platform can also connect citizens with their government using smart city concepts, allowing citizens to access information about health, environment, energy use, and city development plans. The geospatial industry expects increasing demand for digital twins and the corresponding opportunities they present. Geospatial data and information are critical enablers for smart cities and digital twins, and the geospatial industry needs to [Extracted from the article]

• Deep features are extracted from the CT images using deep convolutional neural network model with dense layers. • Hyper edges are constructed for each images by using extracted deep features. A novel hypergraph convolutional neural network is constructed and trained by representational learning. • Evaluated the performance of the trained neural network with test data and attained ideal accuracy, precision, sensitivity, specificity, recall and F1-score. Kidney failure, also called as end-stage renal disease is the final stage of chronic kidney disease(CKD). Kidney failure cannot be revered and is the life-threatening if left untreated. Hence there is a demand for the early and automatic detection of CKD from radiology images based on deep learning techniques. This work focuses on the diagnosis of the predominant CKDs such as kidney stone, cyst and tumor from CT scan images. CT scan abdomen and urogram images are used to construct the digital-twin for the diagnosis of CKD in the robust manner. The proposed work is based on the dataset which contains 12,446 unique CT scan images, within it in which the cyst contains 3,709, normal 5,077, stone 1,377, and tumor 2,283. Deep features are extracted from these images and generated hyper edges from these features. These features are used to construct hypergraph representing the renal images. These hypergraphs are used in hypergraph convolutional neural network for representational learning. Finally, the model is validated by hold-out dataset. Deep learning metrics including precision, recall, accuracy, and FI score are used to validate the proposed model. It outperforms compared to other state of art algorithms with a superior validation accuracy of 99.71. The proposed model is a robust digital-twin for the early diagnosis of kidney diseases. It will assist the nephrologist for the better prognosis of kidney related abnormalities. [ABSTRACT FROM AUTHOR]

This work proposes a new method for the detection, localization, and classification of grid-connected photovoltaic (PV) array faults. Line-to-line, open-module, shorted-module, open-string, and shorted-string faults as well as partial shading conditions are studied. The proposed method has two stages, which are (1) detection and localization of faults and (2) classification of faults. In the first stage, detection and localization are performed using a digital twin (DT) by analyzing the current ratio of each PV array. The measured DC (direct current) power after the operation of the DC/DC boost converter in the physical object is firstly converted into a 2-dimensional image using a recurrence plot (RP) and is then inputted to the second stage. A deep learning-based shifted windows (swin) transformer optimized by particle swarm optimization (PSO) is used in the classification stage, eliminating the need for model tuning by trial-and-error. A PV system of ten arrays with 49 kW is studied. The coefficients of determination (R2) between the results of the digital object (digital twin) and the physical object for different scenarios demonstrate the accuracy and success of the digital twin. R2 values of 0.99988 for varying irradiation with constant temperature and 0.97923 for constant irradiation with varying temperature indicate strong correlations between the digital and physical objects, further confirming the applicability of the digital twin in PV fault detection. The comparative evaluation of the PSO-optimized swin transformer against classical machine learning algorithms and state-of-the-art convolutional neural networks reveals the superior performance of the proposed method. It achieves an outstanding classification accuracy of 98.55%, demonstrating its ability to effectively classify various types of PV faults. The results of the area under the curve (AUC) of the receiver operating characteristic (ROC) curve, which measures the trade-off between true positive rate (TPR) and false positive rate (FPR), further illustrate the effectiveness of the proposed method for PV fault classification. • A model-based digital twin is used to detect and locate faults in a PV system. • Shifted Window Transformer with few trained variables is used to classify PV faults. • Topology of deep learning model is optimized by particle swarm optimization. [ABSTRACT FROM AUTHOR]

A digital twin framework is presented for assembly systems with compliant parts fusing sensors with deep learning and CAE simulations. Its underlying concept, 'process capability space,' updates iteratively during evolving tasks of new product introduction with resulting model fidelity able to simulate dimensional, geometric and weld quality of parts and assemblies; isolate root causes of quality defects; and, suggest corrective actions for automatic defects mitigation; thereby, enabling 'Closed-Loop In-Process (CLIP) quality improvement' during assembly system development. Results, using the first fully digitally developed remote laser welding process for aluminium doors, yielded a right-first-time rate of >96% for door assembly cell development. [ABSTRACT FROM AUTHOR]

Preparing the infrastructure for analyzing, recognizing, and characterizing microorganisms in the Metaverse can transform the fields of biology, medicine, and drug discovery. Accordingly, the realization of digital twins of microorganisms, such as viruses, fungi, algae, bacteria, protozoa, archaea, and multicellular animal parasites, can streamline the applicability of the Metaverse and similar emerging technologies like Cyber-Physical Healthcare Systems (CPHS). This is why a new approach to the digital twinning of bacteria has been presented in this research. This method of digital twinning can revolutionize the research and study of bacteria because it allows us to separate useful and harmful bacterial species, increasing the efficiency of treatment noticeably. This innovative method can easily be used by clinics, medical centers, or even by private users physically and virtually and can be adapted to every centralized or decentralized Metaverse Platform. To determine the proper treatment, biologists have always tried to identify the correct bacterial species that prompted a bacterial infection. They use various indicators, such as the bacterial cell's shape and the size of the colony formed by the bacteria, to classify different types of bacteria with different biochemistries and shapes. However, it is challenging because of the extensive similarities between some species. For instance, such similarities exist between Staphylococcus aureus and Staphylococcus saprophyticus, which has caused numerous false diagnostic reports by operators. Wrong species reports bring about treatment failure and increase antibiotic resistance issues. Therefore, the digital twins of bacteria cover all of their identifiable characteristics and overcome many limitations to study them. In this approach, DTL techniques, including MobileNetV2, EfficientNetV2-S, and ResNet-50, have been employed to build digital twins of bacteria species. A hybrid dataset was used for training and evaluation. Among the models, EfficientNetV2-S exhibited the best performance, with a validation accuracy of 99.58% and a test accuracy of 99.33%. The results showed the ability of deep learning models to make bacteria digital twins based on image processing from different labs, thus assisting experts in speeding up the process and reducing diagnostic errors. In addition, in the mispredicted cases, the correct species was among the first three choices of the model. Therefore, not only can experts use DTL approaches to speed up the digital twin realization of microorganisms in the Metaverse, but they can also use these methods to reduce diagnostic errors. • Introducing a technique to streamline digital twinning of microorganisms using CNN. • Describing machine learning digital twins of microorganisms in the Metaverse. • Presenting an exact, economical digital twinning approach via smartphone photography. • Providing a comprehensive bacteria species dataset for future method development. [ABSTRACT FROM AUTHOR]

• Examined essential data-acquisition tools to maintain citizen-centric digital twins (CCDT). • Investigated the machine learning taxonomy for CCDT that broadly classifies into five categories. • Evaluated the performance of selected application processing interfaces for CCDT. • Created use cases for integrating stakeholders' feedback regarding infrastructure and its service buildings. • Unveiled noticeable gaps impeding the development of CCDT and future directions. Serious interoperability challenges prevent the stakeholders of infrastructure projects and citizens as the final users, from interacting with each other and helping maintain a project over its lifetime. This paper focuses on macro-scale digital twins collecting stakeholders' feedback as the end users of the infrastructure and its service buildings. The aim is to examine various technologies for developing a CCDT and use the information processed to maintain and manage infrastructure services. This involves a systematic review, investigating technologies for data acquisition, data processing, and interface development to improve CCDT capabilities. Among the 89 selected articles, 16% of the sample dataset directly focused on users' engagement. When considering data acquisition technologies, the open data platforms (37% of the sample dataset), remote sensors (37%), and IoT sensors (8%) ensure the dynamic capabilities of the digital twin. Volunteered geographic information (VGI) and social sensing are two prominent technologies that encourage citizen engagement. The number of articles considering the use of segmentation and classification and object detection and tracking algorithms at city-scale digital twins is significant, accounting for 25% and 24% of all articles discussing various algorithms. Further, the study carried out a comprehensive analysis of application programming interfaces (APIs) while presenting their specifications, features, and applications. [ABSTRACT FROM AUTHOR]

• We develop a clinically interpretable dual-stream AI model for the automatic diagnosis of fetal distress. • We combine clinical knowledge traction to quantize numerous clinical diagnostic indicators of authoritative guidelines. • We employ the TreeSHAP-based white box interpreter and CNN-based case tracking model to enhance clinical interpretability. • Our approach achieves a high precision pathological diagnosis of 0.963, which provides solutions for future AI-aided diagnosis systems. Clinically, Fetal Heart Rate (FHR)-based intelligent cardiotocography classification to diagnose fetal well-being is of utmost importance for obstetricians and gynecologists. However, the current Fetal Distress Diagnosis (FDD) algorithms based on Artificial Intelligence (AI) methods are designed as black box models that usually result in poor performance on clinical interpretation and lack explicit diagnostic evidence. This paper focuses on two aspects: enhancing the accuracy of FD and providing explanations for the model's decisions. Specifically, we propose a novel clinically interpretable dual-stream AI architecture for FDD. The two streams embed multi-feature category representations and the original FHR into the Digital Twin model (DT) and the deep convolutional neural Networks-based Case Tracking model (CTNet), respectively, thus marking as DT-CTNet. And the former makes the input-outcome dependency transparent, while the latter provides the samples that are the most similar to the object being diagnosed. By this design, not only is high-precision FDD achievable, but a transparent explanation with global, local and instantiation interpretation can be obtained. To the best of our knowledge, this work is the first to provide an interpretable diagnosis of fetal distress. The proposed method is comprehensively evaluated from five aspects: classification model accuracy, explanation of the classification model decision, consistency of the evidence explanation, pathological difference in visual analysis, and a comparison with state-of-the-art methods. Experimental results, which are performed on a challenging dataset, show that the proposed fetal distress diagnosis method outperforms the state-of-the-art methods and has a high accuracy (0.963). [ABSTRACT FROM AUTHOR]

The Industrial 5.0 Model integrates enabling technologies such as deep learning, digital twins, and the meta-universe with new development concepts. However, model and data security may pose challenges for developing zero-defect production and other industrial manufacturing industries. To address this issue, we generate adversarial examples using a one-pixel attack in adversarial machine learning, which can fool the defect detection classification model. The traditional one-pixel attack based on the Differential Evolution (DE) algorithm has limited global search ability. Therefore, we use a novel algorithm called T eaching and L earning-based M oth- F lame O ptimization (TLMFO), which enhances the global search performance and improves the attack effectiveness. We evaluate TLMFO on benchmark functions and attacks on Cifar10 and ImageNet datasets, and compare it with MFO and DE. The results show that TLMFO outperforms both MFO and DE in terms of accuracy and speed of convergence. Moreover, TLMFO achieves notably better attack effectiveness than DE under targeted and untargeted attacks on the Cifar10 dataset and under-targeted attacks on the ImageNet dataset. Our research confirms that safety prevention is a link worth considering in developing Industry 5.0. [ABSTRACT FROM AUTHOR]

Natural gas jet fire induced by igniting blowouts has the potential to cause critical structure damage and great casualties of offshore platforms. Real-time natural gas jet fire plume prediction is essential to support the emergency planning to mitigate subsequent damage consequence and ocean pollution. Deep learning based on a large amount of Computational fluid dynamics (CFD) simulations has recently been applied to real-time fire modeling. However, existing approaches based on point-estimation theory are 'over-confident' when prediction deficiency exists, which reduce robustness and accuracy for emergency planning support. This study proposes probabilistic deep learning approach for real-time natural gas jet fire consequence modeling by integrating variational Bayesian inference with deep learning. Numerical model of natural gas jet fire from offshore platform is built and the natural gas jet fire scenarios are simulated to construct the benchmark dataset. Sensitivity analysis of pre-defined parameters such as MC (Monte Carlo) sampling number m and dropout probability p is conducted to determine the trade-off between model's accuracy and efficiency. The results demonstrated our model exhibits competitive accuracy with R 2 = 0.965 and real-time capacity with an inference time of 12 ms. In addition, the predicted spatial uncertainty corresponding to spatial jet fire flame plume provides more comprehensive and reliable support for the following mitigation decision-makings compared to the state-of-the-art point-estimation based deep learning model. This study provides a robust alternative for constructing a digital twin of fire and explosion associated emergency management on offshore platforms. • Advanced deep probabilistic learning-based real-time natural gas jet fire prediction model of offshore platform is proposed. • Proposed model integrates variational Bayesian inference with deep learning backbone. • Optimal variational hyperparameters of proposed model is determined. • Model provides more robust fire consequence estimation of offshore platform compared to state-of-the-art models. [ABSTRACT FROM AUTHOR]

Conventional sidewalk studies focused on quantitative analysis of sidewalk walkability at a large scale which cannot capture the dynamic interactions between the environment and individual factors. Embracing the idea of Tech for Social Good , Urban Digital Twins seek AI-empowered approaches to bridge humans with digitally-mediated technologies to enhance their prediction ability. We employ GraphSAGE-LSTM, a geo-spatial artificial intelligence (GeoAI) framework on crowdsourced data and computer vision to predict human comfort on the sidewalks. Conceptualising the pedestrians and their interactions with surrounding built and unbuilt environments as human-centric dynamic graphs, our model captures such spatio-temporal variations given by the sequential movements of human walking, enabling the GraphSAGE-LSTM to be spatio-temporal-explicit. Our experiments suggest that the proposed model provides higher accuracy by more than 20% than a traditional machine learning model and two state-of-art deep learning frameworks, thus, enhancing the prediction power of Urban Digital Twin. The source code for the model is shared openly on GitHub. [Display omitted] • A spatio-temporal-explicit GeoAI to predict human outdoor comfort. • Integrating crowdsourced data into conventional comfort studies. • Introducing human-centric computational models for urban sidewalks. • Enhancing the prediction power of Urban Digital Twin. • Envisioning engineering-driven digital twin models to embrace human aspects. [ABSTRACT FROM AUTHOR]