Search

Forward–backward doubly stochastic differential equations (FBDSDEs) serve as a probabilistic interpretation of stochastic partial differential equations (SPDEs) with diverse applications. Coupled FBDSDEs encounter numerous challenges in numerical approximation compared to forward–backward stochastic differential equations (FBSDEs) and decoupled FBDSDEs, including ensuring the measurability of the numerical solutions, accounting for the mutual influences between forward and backward processes, and considering the relationship with respect to SPDEs rather than PDEs. This paper introduces, for the first time, a numerical method for solving multi-dimensional coupled FBDSDEs. By integrating an optimal control-based approach with deep neural networks, it effectively addresses the coupling-related challenges between forward and backward equations. Computational examples of coupled FBDSDEs with explicit solutions demonstrate that the proposed deep learning-based numerical algorithm achieves commendable performance in terms of both accuracy and efficiency. [ABSTRACT FROM AUTHOR]

The main concept of a smart city is to join manual items with electronics, software, sensors, and network connectivity for data contact via Internet of Things (IoT) gadgets. IoT improves production and efficiency by cleverly utilizing remote management, but security as well as privacy risk, rises. Cyber threats are progressing every day, affecting inadequate actions of safety and privacy. In a smart city, the exposed action of an individual or business can keep the whole city in danger. Owing to the trust of numerous modules of smart cities on data and communication techniques, cybersecurity tasks like information leakage and mischievous cyber-attacks. Malicious Activity Detection in the area of cybersecurity for smart cities is dominant in protecting the consistent organization of urban atmospheres. Leveraging innovative machine learning (ML) and deep learning (DL) approaches, mainly anomaly detection methods permits the identification of abnormal patterns and probable threats within the massive streams of data produced by smart city methods. So, this paper designs a Fusion of Optimization Algorithms and DL for Enhanced Malicious Activity Recognition (FOADL-EMAR) technique for cybersecurity in smart cities. The purpose of the FOADL-EMAR technique is to recognize the presence of malicious activities using optimal DL techniques in the environment of a smart city. To accomplish this, the FOADL-EMAR technique uses a Z-score data normalization model to measure the input dataset into a useful format. In addition, the FOADL-EMAR technique uses an improved sand cat swarm optimization (ISCSO) algorithm that can be applied to select an optimal subset of features. An ensemble of three DL techniques is used namely long short-term memory (LSTM), deep neural networks (DNNs), and extreme learning machine (ELM) for malicious activity detection. The Harris Hawks Optimization (HHO) approach has been applied to improve the detection results of the ensemble models. The simulation analysis of the FOADL-EMAR methodology occurs utilizing a benchmark database. The experimentation values demonstrate the improved detection outcomes of the FOADL-EMAR algorithm over other existing methods in terms of different measures. [ABSTRACT FROM AUTHOR]

In this paper, we consider the evolution of a new probability distribution called the exponential flexible Weibull distribution. The proposed exponential flexible Weibull distribution is obtained by combining the flexible Weibull extension with the exponential T- X strategy. For the exponential flexible Weibull, the estimators of the model parameters are derived mathematically. The appraisal of these parameters is accomplished through a simulation study. The applicability and virtuoso of the exponential flexible Weibull distribution are exemplified via two data sets. Furthermore, we implement the state-of-the-art deep learning algorithms, specifically Artificial Neural Networks and Extreme Gradient Boosting, which are widely employed in real-world applications. A comprehensive comparative analysis is conducted to assess the relative performance of these methodologies. [ABSTRACT FROM AUTHOR]

Rolling bearings constitute one of the most vital components in mechanical equipment, monitoring and diagnosing the condition of rolling bearings is essential to ensure safe operation. In actual production, the collected fault signals typically contain noise and cannot be accurately identified. In the paper, stochastic resonance (SR) is introduced into a spiking neural network (SNN) as a feature enhancement method for fault signals with varying noise intensities, combining deep learning with SR to enhance classification accuracy. The output signal-to-noise ratio(SNR) can be enhanced with the SR effect when the noise-affected fault signal input into neurons. Validation of the method is carried out through experiments on the CWRU dataset, achieving classification accuracy of 99.9%. In high-noise environments, with SNR equal to −8 dB, SRDNs achieve over 92% accuracy, exhibiting better robustness and adaptability. • Stochastic resonance is combined with spiking neural networks to improve accuracy. • Diverse neurons are employed in Stochastic Resonance Deep Residual Networks (SRDN). • SRDN exhibits superb robustness and adaptability under different noise conditions. [ABSTRACT FROM AUTHOR]

Moving beyond classical single-label classification tasks proves to be crucial nowadays since the majority of real-life problems are inherently multi-label. Various tasks require involving, for one single instance as an input, several classes simultaneously or none of them. Particularly, detecting abusive texts in Arabic social networks intuitively carries several manifestations with regard to the targets' referred characteristics like gender, race, sexual orientation, etc. Existing multi-label algorithms either invest considerable complexity to handle labels' correlation or explicitly ignore this information to deal with each label as an independent binary problem. To remedy the abovementioned problems, this paper proposes a new multi-label classification solution based on deep neural networks. The main idea is to exploit the correlation between all the dataset's labels to break it into ordered subsets. Then, follow that order to implement a chain of BERT classifiers. Each classifier is therefore trained on a corresponding subset and passes its output via feature space to the following one. For a new instance, we gather all the classifiers' results and pick the best one for each class. Experiments showed that our approach significantly outperformed the single classifier's method using BERT as well as other neural network architectures on our Arabic abusive language dataset. 91%,76%, and 82% were achieved respectively for precision, recall, and F1-score. Moreover, we further investigated the performance of our approach on Kaggle multi-label research paper dataset and achieved very promising results. [ABSTRACT FROM AUTHOR]

• Review of papers on the diagnosis of brain diseases using the fusion of neuroimaging modalities is presented. • For each brain disease, the most appropriate neuroimaging modalities and best combinations are introduced. • Advantages and disadvantages of fusion levels and different modalities are introduced. • Details of various types of deep neural networks used for the task at hand are also examined. • Finally, future directions and challenges in the field are discussed. Brain diseases, including tumors and mental and neurological disorders, seriously threaten the health and well-being of millions of people worldwide. Structural and functional neuroimaging modalities are commonly used by physicians to aid the diagnosis of brain diseases. In clinical settings, specialist doctors typically fuse the magnetic resonance imaging (MRI) data with other neuroimaging modalities for brain disease detection. As these two approaches offer complementary information, fusing these neuroimaging modalities helps physicians accurately diagnose brain diseases. Typically, fusion is performed between a functional and a structural neuroimaging modality. Because the functional modality can complement the structural modality information, thus improving the performance for the diagnosis of brain diseases by specialists. However, analyzing the fusion of neuroimaging modalities is difficult for specialist doctors. Deep Learning (DL) is a branch of artificial intelligence that has shown superior performances compared to more conventional methods in tasks such as brain disease detection from neuroimaging modalities. This work presents a comprehensive review paper in the field of brain disease detection from the fusion of neuroimaging modalities using DL models like convolutional neural networks (CNNs), recurrent neural networks (RNNs), pretrained, generative adversarial networks (GANs), and Autoencoders (AEs). First, neuroimaging modalities and the need for fusion are discussed. Then, review papers published in the field of neuroimaging multimodalities using AI techniques are explored. Moreover, fusion levels based on DL methods, including input, layer, and decision, with related studies conducted on diagnosing brain diseases, are discussed. Other sections present the most important challenges for diagnosing brain diseases from the fusion of neuroimaging modalities. In the discussion section, the details of previous research on the fusion of neuroimaging modalities based on MRI and DL models are reported. In the following, the most important future directions include Datasets, DA, imbalanced data, DL models, explainable AI, and hardware resources are presented. Finally, the main findings of this study are presented in the conclusion section. [ABSTRACT FROM AUTHOR]

This paper introduces a deep learning (DL)-based Automatic Modulation Classification (AMC) model. Our model is considered to be a receiver with a modulation classifier that is capable of differentiating ten modulation techniques. The classifier combines the residual neural network (ResNet) and the long short-term memory network (LSTM). The ResNet boosts the accuracy in deep neural networks, and LSTM improves the classifier's performance by passing the time-series previous state information to the current state. This paper demonstrates that the proposed model achieves 92% peak recognition accuracy at 18 dB SNR. It is higher than the ResNet by 11.4%, the CNN network by 4.7%, and the CLDNN network by 2%. Moreover, it delivers more than 90% classification accuracy at SNR above 0 dB. Additionally, it improves the classification accuracy at low SNR by achieving 85.5% accuracy at −2 dB SNR. Furthermore, it advances the recognition accuracy of various modulation recognition methods by more than 98% at SNR above 0 dB. [ABSTRACT FROM AUTHOR]

Multi-view learning (MVL) is a rapidly evolving direction in the field of machine learning. Despite the positive results, most algorithms that combine multi-view learning with twin support vector machines (TSVM) focus on the traditional machine learning domain. No method has been accomplished for combining MVL, TSVM, and deep learning. In this paper, we propose two novel multi-view deep models to solve the multiclass classification problem, namely deep multi-view twin support vector machines (DMvTSVM) based on deep neural network (DNN) and auto-encoder (AE) network. They find two non-parallel hyperplanes such that each hyperplane is as close to its own class as possible while being as far away from the other class as possible. Meanwhile, we apply similarity regularization to the output of the Deep TSVM classifier for each view to learn consensus information between views, and use this to refine the joint weights of the deep model and TSVM. Finally, the novel models employ the o n e − v s − r e s t strategy to allow the DMvTSVM classifier to solve the multiclass classification problems. In the experiments, the proposed methods are compared with existing state-of-the-art algorithms to prove their effectiveness. • The first combination of twin support vector machines (TSVM) and deep neural networks. • Nesting of more advanced auto encoder networks. • Incorporating multi-view learning capabilities. • Mutual negotiation and simultaneous learning between deep neural networks and TSVM. • The experiments prove the effectiveness of the algorithm in this paper. [ABSTRACT FROM AUTHOR]

Transistor scaling has continued to yield significant performance gains. The general-purpose architecture will not suffice in this dark silicon era with the big data boom and growing emphasis on performance. Artificial neural networks and deep learning are sophisticated computing activities that cannot be optimally executed (accuracy, performance, and energy) on today's general-purpose computer architecture. The demand for domain-specific computer architectures is increasing. This paper suggests and evaluates a highly reconfigurable domain-specific design for deep learning and artificial neural networks to overcome these problems. An FPGA-based soft-core processor for deep neural network (DNN) acceleration is suggested in this paper. Using a combination of unicast and multicast for collective operations, the proposed FPGA accelerates the performance of the deep learning network. The suggested design outperforms the current general-purpose processor by 6.6 times, according to a thorough examination and comparison. The existing architecture's level of accuracy is retained. [ABSTRACT FROM AUTHOR]

As a vital mechanical sub-component, the health monitoring of rolling bearings is important. Vibration signal analysis is a commonly used approach for fault diagnosis of bearings. Nevertheless, the collected vibration signals cannot avoid interference from noises which has a negative influence on fault diagnosis. Thus, denoising needs to be utilized as an essential step of vibration signal processing. Traditional denoising methods need expert knowledge to select hyperparameters. And data-driven methods based on deep learning lack interpretability and a clear justification for the design of architecture in a "black-box" deep neural network. An approach to systematically design neural networks is by unrolling algorithms, such as learned iterative soft-thresholding (LISTA). In this paper, the multi-layer convolutional LISTA (ML-CLISTA) algorithm is derived by embedding a designed multi-layer sparse coder to the convolutional extension of LISTA. Then the multi-layer convolutional dictionary learning (ML-CDL) network for mechanical vibration signal denoising is proposed by unrolling ML-CLISTA. By combining ML-CDL network with a classifier, the proposed denoising method is applied to the explainable rolling bearing fault diagnosis. The experiments on two bearing datasets show the superiority of the ML-CDL network over other typical denoising methods. • The multi-layer convolutional LISTA (ML-CLISTA) algorithm is derived. • The multi-layer convolutional dictionary learning network is proposed. • The proposed method is applied to the explainable rolling bearing fault diagnosis. [ABSTRACT FROM AUTHOR]

A key development towards enhancing computer-human interaction is emotion recognition. This publication describes a technique called EmoCNN, which uses deep learning techniques to precisely identify and classify human emotions, emphasizing improving model performance using different optimizers. Our research intends to contribute to the creation of more effective systems that improve computer-human interaction by solving the problems associated with emotion recognition. By bridging the gap between humans and robots, accurate emotion detection enables systems to perceive emotions for customized and responsive interactions. AI-powered assistants, chatbots, and social robots all benefit from emotion recognition by providing more responsive, empathic and interesting user experiences. Emotion-aware technologies can also enhance user feedback analysis, human-centered design, and monitoring of mental health. Using a human emotion detection dataset, we carried out comprehensive experiments focusing on the happy, sad, and neutral emotion classes. Constructing a customized EmoCNN model with convolutional layers, a hidden layer, ReLU activation, and max-pooling was the focus of our computational work. We investigated various optimizers and evaluated how they affected accuracy, convergence speed and loss minimization. The results demonstrated that the EmoCNN model, which had been trained using the Adam optimizer, gave the best accuracy in distinguishing between emotions. Our paper provides a comparative analysis, highlighting the superiority of EmoCNN over existing models, showcasing its ability to achieve higher validation accuracy (89%) and more efficient emotion recognition when compared to previous approaches with minimal loss. Our research advances the field of emotional computing by demonstrating how well EmoCNN can identify and categorizes various human emotions. This discovery has significant ramifications for the creation of emotion-aware computers, which can better understand and react to human emotions, enhancing computer-human interaction. [ABSTRACT FROM AUTHOR]

Non-contact ultrasonic sensors are generally used for range measurement and object detection. In addition to the shape and size of the target object, the identification of the material types plays a vital role in robotic navigation and autonomous vehicle applications. Ultrasonic echo signals contain a significant amount of information and can be used to recognize and categorize different materials. Echo signals can help robots detect the obstacle's material on the path and comply with its behavior accordingly. The emergence of deep neural networks has shown great promise, offering cutting-edge performance for a wide range of signal-processing tasks. This work uses the non-contact ultrasonic echo signal from a set of materials (glass, wood, metal plate, sponge, and cloth) for classification. The main idea is to classify the materials using the embedded information of reflected echo signals. Hilbert transform is used to get the envelope from the raw echo ultrasonic signal. In this paper, a novel architecture for one-dimensional convolutional neural network (1D-CNN) has been proposed to accomplish the classification task. The CNN model takes raw echo signals as input to detect and classify materials accurately. The performance of the classifier model is evaluated using accuracy, precision, recall, and F1-score. The proposed 1D-CNN deep learning-based multi-class classifier model can classify the different types of materials with an accuracy of 96%, precision of 95%, recall of 95%, and F1-score of 95%. [ABSTRACT FROM AUTHOR]

Autonomous routing through curved path for point-to-point mobile robot is a well-established challenging problem in the area of intelligent path navigation. Point-to-point collision-free locomotion of AGV (Automated Guided Vehicle) requires precise differentiation of movable path from other environmental visual noise. As these path navigating depend on various visual cues and precepts, naturally the semantic meaning of path in contrast to other closely available objects on the visual window requires significant amount of understanding of the input from precepts. In this paper, it is experimentally established that semantic segmentation has proved to be a concise method in making comparative better segregation of paths from midway structured obstacles that are with high computational cost. The claims are also evident from surveyed works. The challenge of reducing high computational performance with efficient segmentation of movable area has been addressed in this research work with a focus on deriving better filtered accessible paths with a degree of algorithmic confidence. Further, this segmented result is taken for curved trajectory tracking by the AGV. A systematic percept mediated deep based Semantic-region Aware Model Predictive Trajectory Tracking(SAMP-TT)Algorithm has been proposed to solve the problem of precise obstacle detection on curved path trajectories. RGB (Red, Green Blue) image of outdoor multi-sectioned area has been collected by Intel Realsense D455 which worked as input feed to the respective neural network models. The performance of the proposed technique has been benchmarked with existing segmentation models including ENET (Efficient Neural Network), U-NET to confirm the considerable reliability and accuracy of the presented algorithm. It has been followed by the state-of-art comparative visual result analysis of different taken models with the proposed architecture. [ABSTRACT FROM AUTHOR]

Stress is a common experience with significant impacts on our health, which has sparked an increase in research aimed at detecting stress. This paper explores the effectiveness of using Electrocardiogram (ECG) signals as a key indicator for stress detection. What distinguishes our research is the carefully constructed architecture of our deep neural network model, coupled with the meticulous optimization strategies employed. We trained and evaluated our model using the comprehensive WESAD[10] dataset, which includes PANAS[10] self-reports. Our proposed model demonstrates remarkable improvements in accuracy and F1 score compared to previous attempts, highlighting the potential of deep learning techniques in understanding and predicting stress levels based on physiological data and harness its power to offer an objective measure of the body's response to stress. The results of the DeepChill demonstrate the effectiveness of the deep learning model in detecting stress from ECG signals, highlighting its potential for health monitoring and intervention. This contributes to the field of affective computing by providing a reliable and efficient method for stress detection. The proposed fully connected neural network model has attained an impressive accuracy of 92% surpassing the 85% accuracy reported on the WESAD dataset. [ABSTRACT FROM AUTHOR]

Mushrooms are fungi that play significant roles in ecosystems, food, and medicine. However, distinguishing toxic mushrooms from non-toxic ones is challenging, as their visual characteristics can be misleading. Deep learning, a subset of artificial intelligence, offers a promising approach to detecting and classifying mushrooms based on their toxicity. By training deep neural networks on datasets of labelled mushroom images, it becomes possible to create models capable of accurately differentiating between toxic and non-toxic mushrooms. Mushrooms are fungi that play significant roles in ecosystems, food, and medicine. However, manually distinguishing toxic mushrooms from non-toxic ones is a challenging and time-consuming task. This research work is to help the local farmers and exporters to identify the difference between the toxic and the edible mushroom which will eventually boost production and will directly have an impact on the economic growth in India. This paper compared different deep learning algorithms such as Support Vector Machine (SVM), Resnet50, YOLO V5 and Alexnet, which have been found to be the most useful in classifying the mushrooms effectively as toxic and non-toxic respectively. The patterns in the images of the mushrooms have been predicted through the above classifiers. The validation techniques show SVM to be the most effective with an accuracy of 83%. [ABSTRACT FROM AUTHOR]

Cybersecurity threats, particularly spam SMS, are increasingly sophisticated, demanding more advanced detection systems. Traditional spam detection methods fall short due to their ineffectiveness against novel threats and lack of transparency. This paper investigates the role of Explainable Artificial Intelligence (XAI) in spam detection, emphasizing the interpretability of AI-driven systems through SHapley Additive explanations (SHAP). We propose a hybrid model combining BERT with Random Forest (RF) and Artificial Neural Networks (ANN) for spam detection, and employ SHAP values to elucidate the decision-making process. The study demonstrates that our XAI approach not only improves the accuracy of spam detection but also enhances the transparency and trustworthiness of the predictions. These findings suggest that the incorporation of XAI into spam detection models is not only beneficial but necessary for future cybersecurity measures. Our research invites further exploration into other XAI techniques and their applications in real-world scenarios. [ABSTRACT FROM AUTHOR]

Epilepsy is a brain disorder that can affect people of all ages which is one of the most common neurological disorders affecting around 50 million individuals. Healthcare Chatbots schedule appointments, respond to patient inquiries, and offer relevant information, freeing medical staff to handle life crises. This paper implements a chatbot for epilepsy patients to increase awareness about epilepsy. Patients may easily access this AI-based healthcare chatbot whenever they have queries regarding precautions to be taken by epilepsy patients. Chatbot uses a rich set of epilepsy-related datasets collected from trusted websites which are preprocessed with NLP techniques to prepare the data for model inputs.DNN, LSTM, and RNN models are created and evaluated to produce answers to user queries related to possible precautions to avoid seizures, safety measures to be taken by patients with seizure history, etc. Evaluation of models shows that the RNN model generates more accurate answers with 94% accuracy. Finally, the epilepsy question answering Chatbot for precaution is concluded with the RNN model. [ABSTRACT FROM AUTHOR]

Diabetic Retinopathy (DR) is an eye ailment in diabetics, often causing vision loss. Timely retinal screenings can help prevent blindness. While DR cannot be cured, early detection can curb vision decline. Ophthalmologists diagnose DR using Optical Coherence Tomography (OCT) or Fundus Images. Evaluating via fundus images can be complex and potentially imprecise. The study offers a computer-supported diagnosis to aid ophthalmologists. It uses Multi-level Otsu thresholding to segment Fundus Images, with segmented outputs processed further. Advanced feature extraction techniques like the Gray-Level Co-occurrence Matrix (GLCM) and dynamic Flemingo optimization enhance DR feature identification. Additionally, a novel cascaded voting ensemble deep neural network model is introduced, merging multiple algorithmic insights to improve classification. The paper concludes by aligning classifications with a standard grading system, giving clinicians a clear DR severity gauge for better treatment planning. [ABSTRACT FROM AUTHOR]

The stock market is a very important part of the financial field, and the prediction of the stock market has a great relationship with the returns and risk safety of the entire financial field. With the continuous mature application of machine learning and deep learning in other fields, such as image processing and text analysis, people begin to focus on the use of different models so as to predict stock volatility. However, in view of the unique multi-source and heterogeneous characteristics of stock information, the artificial neural network relying on deep learning cannot make a good prediction on it. At this time, the graph neural network that can well analyze the graph structure data is gradually favored by scholars at home and abroad, and the research thinking is also expanding. This dissertation examines the purpose of deeply analyzing the methods of different graph neural network models on stock prediction through an inductive study of amount of relevant literature. In this paper, we not only classify the literature by various graph neural network models, but also describe objectively the models and ideas presented in each paper. By referring to literature, this paper summarizes the previous research results, analyzes the applicability and results of different methods, and lays a foundation for better stock prediction in the future. [ABSTRACT FROM AUTHOR]

The emergence of artificial intelligence (AI) and its related capabilities has led industries to rethink the existing practices of conventional supply chain management and data analysis. Machine learning (ML), Deep Learning (DL) and their unique ability to predict future data and classify data have led to important research in the supply chain (SC) domain, particularly in identifying and prioritizing supply chain risks. This paper proposes several DL methodologies to exploit the benefit of DL, particularly to identify whether any product will be delivered late due to any unforeseen reason in a complex SC system. Four different DL architectures (Simple-LSTM, Deep-LSTM, 1D-CNN, and TCN-1DSPCNN models) are proposed to extract features, while six variant classifiers: Softmax, random trees (RT), random forest (RF), K-nearest neighbor (KNN), artificial neural network (ANN), and support vector machine (SVM), were used to classify delay or non-delay information. By seamlessly capturing intricate temporal dependencies, these DL models enhance accuracy in robustly identifying supply chain late orders. Leveraging their hierarchical feature learning, these proposed DL models excel in recognizing subtle patterns and correlations, making them ideal for classifying late orders within the supply chain. Their parallel processing prowess facilitates real-time decision support, allowing organizations to address potential delays and allocate resources effectively and proactively. Five-fold cross-validation is presented to avoid over-fitting and to prove the efficiency of the proposed DL models. The total accuracies of the six ML classifiers are 74.03, 75.81, 93.35, 87.72, 93.59, and 95.10, respectively, while the maximum accuracies obtained from four proposed DL methodologies obtained an accuracy of 97.6, 98.63, 100, 100% respectively using the SVM classifier for predicting late orders based on five-fold cross-validation. • This paper investigates a few DL approaches to extract features of SC data. • Both RNN and CNN are applied in the same model. • An improved CNN model has been proposed for feature extraction. • An online dataset is employed to validate the proposed DL architectures. [ABSTRACT FROM AUTHOR]

In the context of early childhood education, students need to acquire fundamental writing skills for their lifelong development. Public schools, especially in low- and middle-income countries, should monitor individual student progress to mitigate the detrimental effects of socioeconomic vulnerabilities in education. Furthermore, the volume of students often overwhelms teachers responsible for assessing handwriting texts and providing feedback. This article conducts a Systematic Literature Review (SLR) focusing on solutions for automatically evaluating students' handwriting, discussing their performance, future research directions, and areas needing further investigation. We used a mixed-methods approach to conduct our SLR, encompassing a search across four databases (ACM Digital Library, IEEE Xplore, ScienceDirect, and SpringerLink) and employed the snowballing technique. We used the initial set of papers from the database search as the foundation for the subsequent snowballing search. Findings revealed that the studies introduced computational techniques, examined or enhanced existing methods, and developed assessment tools. These solutions predominantly employed techniques such as artificial neural networks and used available datasets comprising handwritten images, answers, or student essays. Only some studies provide low-cost solutions for automatically assessing the writing abilities of underserved public school students. • We reviewed 491 papers, extracting data from 22 focusing on handwriting assessment. • We applied a mixed method using database search and the snowballing technique. • Studies frequently adopt deep learning methods to tackle the handwriting recognition problem. • Studies employ various methods to automatically assess text production, such as ANN, latent semantic analysis, content vector analysis, and CNN. • The mean accuracy for handwriting assessment was notably high at 93.64%, suggesting that the models in this group exhibited good performance. [ABSTRACT FROM AUTHOR]

The main objective of this paper is to describe a methodology that was developed to support maintenance decision-making methods based on equipment condition. Condition-Based Maintenance allows to increase equipment availability and maximize investments. This is mainly due to the prevention of unexpected equipment downtime. By avoiding turning on/off industrial equipment, production flows are more efficient, allowing manufacturers to improve the quality of the end-product. The industry aims more and more to correspond satisfying customer expectations. We argue that the methodology developed in this paper adds value to the existing literature, namely because the fact that it is possible to anticipate the state of an equipment without a large amount of support data. In other words, although one could find information gaps regarding the occurrence of failures, it was possible to accurately assess the state of the equipment. This approach is robust, as it can be used in distinct equipment with different sensors, making this methodology generalizable for Condition-Based Maintenance. The paper presents the validation of the preceding through a case study on drying presses in the paper industry. To do so, three states were adopted, namely: "Proper function"; "Alert state"; and "Equipment failure". The methodology follows a series of steps, going through the collection of values from vibration sensors, imputation of values using Deep Artificial Neural Networks through on-line sensors, until reaching the last stage of classification carried out by the Hidden Markov Model. Through optimized observations from the previous steps, it was possible to define the hidden states through the Viterbi algorithm, which corresponds to the health states of the equipment. Additionally, it was possible to demonstrate that the proposed methodology can accurately characterize the condition states of the equipment based on the data obtained and can be generalized to other types of equipment. • Method classifies equipment status without prior info and predicts anomalies using ML. • Using DNN, time series data management was added to better diagnose equipment condition. • Generic method requires no technical knowledge, works online for real-time predictive maintenance. • The algorithm is generic and applicable to various equipment with different sensors. [ABSTRACT FROM AUTHOR]

Temperature reconstruction is vital for ensuring system reliability in electronic component design. However, current approaches struggle to effectively explore system information and physical relationships, thereby limiting their performance. This paper presents deep learning surrogate models for precise temperature field reconstruction, showcasing their effective discernment of system distribution laws. However, the scarcity of high-quality training data poses a significant challenge, often leading to issues like overfitting and compromised precision. To address this problem, the paper proposes an adaptive multi-source information fusion method (MFIF) for integrating physical information from various data sources in the frequency domain. By leveraging frequency domain analysis, a deeper understanding of underlying physical phenomena is achieved, facilitating effective integration of information. Furthermore, by utilizing deep surrogate models and high-quality training samples, the developed multi-source frequency fusion method enables the creation of a multi-source fusion driven deep learning method for temperature field reconstruction. The proposed method enhances the robustness, accuracy, and effectiveness of aircraft temperature field reconstruction in orbit. Experimental results demonstrate a substantial decrease in both noise and errors, while the Signal-to-Noise Ratio can be improved by up to more than 86%. [ABSTRACT FROM AUTHOR]

Civil infrastructure relies heavily on structural health monitoring systems. However, these systems often encounter challenges due to sensor failures and environmental damage. Consequently, numerous anomalous data points are generated, significantly distorting the accuracy of structural safety assessments. While deep neural networks have emerged as a promising tool for efficiently identifying abnormal data, the meticulous optimization of hyperparameters during training remains a challenge. To address this challenge, this paper introduces a novel approach termed multiple transfer learning, designed to continually enhance a model's classification performance without the need for meticulous hyperparameter configurations. This methodology achieves adaptive training by iteratively migrating across bridge anomaly datasets, bypassing the need for elaborate hyperparameter setting. In this study, five distinct hyperparameter working conditions are established and evaluated to validate the effectiveness of the multiple transfer learning method. The findings highlight the robustness of this approach, demonstrating that multiple transfer learning achieves satisfactory recognition accuracy levels irrespective of the initial hyperparameter setting during network model training. This method circumvents the need for continuous hyperparameters optimization, enabling the adaptive detection of abnormal bridge data. • A training method called multiple transfer learning based on transfer learning is proposed in this paper to solve setting hyperparameters. • The relationship between the number of multiple transfer learning times and the accuracy of anomaly data identification is displayed. • Comparison experiments with different working conditions were set up to verify the effectiveness of multiple transfer learning. [ABSTRACT FROM AUTHOR]

The objective of this paper is to optimize the energy consumption and performance of fuel cell electric bicycles (FCEBs) under specific key input parameters. The paper applied an integrated method that includes an artificial neural network (ANN) and genetic algorithm (GA) to forecast and identify an optimal performance and energy consumption of FCEBs. The simulation model of FCEBs is established and simulated in MATLAB-Simulink environment to generate 1000 data points, that are used for training, validating, testing artificial neural network, the ANN architecture containing five input neurons, two hidden neurons and two output neurons, respectively. Furthermore, the GA is integrated to find the maximum performance and energy consumption once the ANN has exactly been trained. The study found that the FCEB configuration can achieve an effective performance at 30.3 km/h with required power of 210.4 W under speed level_5, radius of wheel 0.39 m, frontal area 0.423 m2, slope grade 0%. In order to validate and verify the simulated results, the experimental approach method was conducted in the same condition. The experimental results fit well with the simulated results in the same initial input parameters. • A method integrating the artificial neural network with genetic algorithm is applied. • The consumed power and fuel cell voltage are optimized by combining artificial neural network and genetic algorithm (ANN-GA). • The effective performance area of fuel cell electric bicycle under various a structure and operating parameters is identified. • Deep-learning results reflect major physical mechanism of fuel cell electric bicycle operation. [ABSTRACT FROM AUTHOR]

The rockfall impact force is important for condition assessment and high-performance structure design of flexible barrier systems, but it is difficult to measure by contact sensors in full-scale tests. To address this issue, this paper developed an automatic and noncontact impact force reconstruction method using deep learning technology. In the proposed method, a high-speed camera was employed to capture the motion images of the whole rockfall impact process, and the motion trajectories of the falling rock were extracted, which were further employed for impact force prediction with a deep learning model. The contributions of this paper lie in two aspects: (1) a highly efficient rockfall motion detection method was presented to extract the two-dimensional (2D) displacements and impact angle of the rockfall using the Swin-transformer neural network; (2) a highly accurate impact force reconstruction method was developed to model the complicated relationship between rockfall trajectory and impact force using a sequence-to-sequence deep learning modeling framework with an attention mechanism. The effectiveness of the proposed method was verified by numerical models and full-scale experiments of two different flexible barrier systems with impact energies of 750 kJ and 2000 kJ. The results show that the extracted rockfall displacements and reconstructed impact forces agree well with reference values with a correlation coefficient above 98%, demonstrating the accuracy and robustness of the proposed method. The proposed method has the unique advantage of remote sensing, which improves the efficiency of emergency response for protection systems and transport infrastructure under rockfall hazards. • A deep neural network was trained for accurately extracting motion trajectories including 2D displacements and impact angle of rockfall. • A sequence-to-sequence deep learning method based on a self-attention mechanism was proposed for impact force reconstruction from noncontact vision-based displacements. • Numerical examples and two full-scale experiments of the flexible barrier system have been conducted to verify the the developed method. [ABSTRACT FROM AUTHOR]

Search and retrieval remains a major research topic in several domains, including computer graphics, computer vision, engineering design, etc. A search engine requires primarily an input search query and a database of items to search from. In engineering, which is the primary context of this paper, the database consists of 3D CAD models, such as washers, pistons, connecting rods, etc. A query from a user is typically in the form of a sketch, which attempts to capture the details of a 3D model. However, sketches have certain typical defects such as gaps, over-drawn portions (multi-strokes), etc. Since the retrieved results are only as good as the input query, sketches need cleaning-up and enhancement for better retrieval results. In this paper, a deep learning approach is proposed to improve or clean the query sketches. Initially, sketches from various categories are analysed in order to understand the many possible defects that may occur. A dataset of cleaned-up or enhanced query sketches is then created based on an understanding of these defects. Consequently, an end-to-end training of a deep neural network is carried out in order to provide a mapping between the defective and the clean sketches. This network takes the defective query sketch as the input and generates a clean or an enhanced query sketch. Qualitative and quantitative comparisons of the proposed approach with other state-of-the-art techniques show that the proposed approach is effective. The results of the search engine are reported using both the defective and enhanced query sketches, and it is shown that using the enhanced query sketches from the developed approach yields improved search results. [Display omitted] • The first learning-based strategy to clean rough query sketches of 3D CAD models • Introduces SketchCleanNet — an end-to-end image translation scheme • SketchCleanNet aims to understand the mapping between rough sketches and clean query images • A novel scheme to calculate the loss is introduced • Dataset Contribution: The resulting enhanced query sketch dataset is made available publicly. • This paper will significantly contribute to the research community and give researchers opportunities to develop new algorithms for search and retrieval of 3D mechanical components. [ABSTRACT FROM AUTHOR]

Diabetic retinopathy (DR) is one of the most common retinal diseases that cause preventable blindness in diabetic patients. The timely screening and grading of retinal images minimize the possibility of vision loss. However, manual screening of retinal images, for detecting micro lesions in the early stages of DR, is time-consuming. This paper proposes an ensemble of deep convolutional neural network (CNN) models for accurate detection and grading of DR using fundus images. Each input image is divided into four patches at the first stage and passed on to pre-trained CNN models (InceptionV3, Xception) for training. The relevant features in the shallow-dense layers of CNN models are utilized as prior knowledge. The integration of shallow and dense layer features helps the model learn the significant information of DR images. At the second stage, a classifier based on artificial neural network is trained using the fused probability vectors of four patches. The results of individual CNN models are combined to generate the final decision in the third stage. This ensemble approach of multi-stage deep learning model improves the overall classification accuracy of diabetic retinopathy grading. Among the five different classification schemes presented in this paper, multistage patch-based deep CNN (MPDCNN), in which local patch-based and holistic details of fundus image are concatenated, provides the best classification accuracy. This ensemble classifier exhibits 96.2 % classification accuracy with fivefold cross-validation. [ABSTRACT FROM AUTHOR]

Two participants with competing objectives are considered in one-to-one orbital pursuit-evasion problem, one of which attempts to satisfy some certain terminal conditions with a minimum cost through its own control while the other player attempts to maximize the specified payoff and avoids entering this terminal range. This paper treats the problem of minimax time to a given final distance, within which the interception between pursuer and evader is defined to occur. To achieve the successful interception with an evader that adopts any unknown maneuvers, a near-optimal interception strategy for the orbital pursuit-evasion is presented in this paper. The pursuit-evasion problem is formulated as a differential game first, and a closed-form barrier solution is provided next. A near-optimal guidance law using deep learning is proposed to intercept the evader inside the capture zone. For the games that start outside the barrier, a learning algorithm for the capture zone embedding strategy is presented based on deep reinforcement learning to help the game state cross the barrier surfaces. The simulation cases of intercepting the evaders that adopt different maneuvering strategies are provided, and the results demonstrate the effectiveness and onboard feasibility of the proposed strategy in solving the orbital interception problem. • The closed-form barrier solution for a planar orbital pursuit-evasion game is provided. • A deep neural network is trained on a dataset consisting of plenty of barrier trajectories generated through the closed-form barrier formula. • The near-optimal guidance law inside capture zone is proposed using trained neural network. • A capture zone embedding strategy is presented based on deep reinforcement learning. • The closed loop interception strategy is proposed combining the strategies outside and inside capture zone. [ABSTRACT FROM AUTHOR]

Thorough maintenance of industrial equipment is crucial for the finances of companies. Whereas the purchase of new tools can be an expensive business, reconditioning special gear often costs just a fraction. In this paper, preliminary steps for an accurate visual based preventive maintenance of hobbing wheels are investigated. To perform robust and reliable decisions about the wheel's condition, tool department specialists require precise taken captures of it. For this reason, a visual control cell is built, which depends on correctly aligned hobbing wheels in its image acquisition construction. The tool needs to be placed on a turn-table and rotated, so that a single tooth is centered in the field-of-view of the camera mounted on a robot arm. For this alignment task, three different main approaches with various preprocessing steps are investigated, a brute-force algorithm, an orb-feature approach and an image regression model. The results show that even a brute-force algorithm can be outperformed by a moderate deep neural network. [ABSTRACT FROM AUTHOR]

Deep learning techniques have shown significant impact in enhancing security across various domains by leveraging artificial neural networks models. When applied to cloud computing security, deep learning offers cost-effective solutions by automating threat detection, reducing manual monitoring, and improving overall security effectiveness. Deep learning models using neural networks play crucial role in tasks like intrusion detection, malware detection, anomaly detection, and log analysis. Integration of deep learning into cloud security requires careful evaluation of existing systems, defining objectives, dataset selection and preparation, model tuning, and eventual modifications for compatibility. Furthermore, implementing deep learning techniques in cloud security entails considering factors such as computational resources, data collection and preparation costs, model development, integration efforts, and ongoing monitoring and maintenance. This paper proposes a feed-forward propagation Artificial Neural Network (ANN) model in cloud security and investigates the key steps for integrating such models into cloud security strategies. Considering that the effectiveness of the ANN model depends on factors such as training data quality, network architecture, and weight adjustment algorithms, the study utilizes a dataset from Kaggle.com for validation and demonstrates steps involved in training and evaluation of the ANN model. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the forward problems on the data-driven rational solitons for the (2+1)-dimensional Kadomtsev–Petviashvili-I (KP-I) equation and spin-nonlinear Schrödinger (spin-NLS) equation via the deep neural networks learning. Moreover, the inverse problems of the (2+1)-dimensional KP-I equation and spin-NLS equation are studied via deep learning. The main idea of the data-driven forward and inverse problems is to use the deep neural networks with the activation function to approximate the solutions of the considered (2+1)-dimensional nonlinear wave equations by optimizing the chosen loss functions related to the considered nonlinear wave equations. • Forward problems of 2D KP-I and spin-NLS equations with rational solitons are studied via deep learning. • Inverse problems of 2D KP-I and spin-NLS equations are discussed via deep learning. • Inverse problems of the 2D combination of NLS and cmKdV equations are explored via deep learning. [ABSTRACT FROM AUTHOR]

The usage of machine learning methods in the financial sector, regarding repayment prediction or forecasting, is quite a new topic, constantly gaining in importance. The concept of the alternative costs in the literature covering machine learning and deep learning occurs most often in connection with the non-financial areas as costs of lost benefits. This empirical paper presents research dedicated to deep learning used in forecasting the alternative costs of leasing represented by the variable KUK_PRC. The study is based on the experimental approach and uses real organization data to solve the forecasting problems in the financial area with AI solutions. This research contributes to the science by identifying and exploration of the research gap in the field of applied economics and finances. The main finding of this paper is the proposed forecasting ACSeq-DNN model that forecasts opportunity costs with smaller deviations from actual values than the forecasting achieved by state-of-the-art models. [ABSTRACT FROM AUTHOR]

The paper presents issues related to methods used in breast histopathological images tumor changes detection. The problem is connected with sustainable health issues which focus on the improvement of health and better delivery of healthcare, rather than late intervention in disease, with resulting benefits to patients and to the environment on which human health depends, thus serving to provide high quality healthcare. The main purpose of the paper is to develop a model based on deep artificial neural networks for the cancer detection in histopathological breast images. The implementation of the proposed model fits in with the concept of sustainable health through the support of the work of doctors in their decisions, diagnosis and in the reduction of the human workload and time, which can be referred to improve the health services. Data set contains 277524 samples from 163 breast histopathological images taken with the WSI scanner. The model is based on a convolutional neural network in the ResNet-18 architecture, which consists of residual blocks. During the final validation on the test set, the network achieved an accuracy of 93.6% and a 87.3% sensitivity in the detection of cancer tissues. The overall performance of the model is characterized by an F1-score of 0.887. The obtained results indicate the possibility of using the system in clinical conditions. [ABSTRACT FROM AUTHOR]

This paper discusses a new approach motivated by the contemporary procedures of short texts' classification by means of deep learning methodology. The method considers the tested collection and the collections of imposters as sequences of short patterns like "tweets". A deep neural network combined with a CNN layer accomplished by the LSTM ones is trained to distinguish between the imposters' collections at the first stage. The obtained network classifies the bathes of the test collection, transforming its texts into signals, making it possible to label them according to their individual writing style. The suggested procedure intends for new attitudes to study textual material applied to recognize authorship attributes of medieval Arabic documents from a novel computational perspective. Evaluating the outcomes proves the method's high reliability demonstrated in the paper on investigating texts questioningly attributed to Al-Ghazali, comparing them with The Holy Quran and the "One Thousand and One Night". The proposed methodology suggests a new look at the perusal of medieval documents' inner structures and possible authorship from the short-patterning and signals processing perspectives. [ABSTRACT FROM AUTHOR]

Weather forecasting is the practice of predicting the state of the atmosphere for a given location based on different weather parameters. Weather forecasts are made by gathering data about the current state of the atmosphere. Accurate weather forecasting has proven to be a challenging task for meteorologists and researchers. Weather information is essential in every facet of life like agriculture, tourism, airport system, mining industry, and power generation. Weather forecasting has now entered the era of Big Data due to the advancement of climate observing systems like satellite meteorological observation and also because of the fast boom in the volume of weather data. So, the traditional computational intelligence models are not adequate to predict the weather accurately. Hence, deep learning-based techniques are employed to process massive datasets that can learn and make predictions more effectively based on past data. The effective implementation of deep learning in various domains has motivated its use in weather forecasting and is a significant development for the weather industry. This paper provides a thorough review of different weather forecasting approaches, along with some publicly available datasets. This paper delivers a precise classification of weather forecasting models and discusses potential future research directions in this area. [ABSTRACT FROM AUTHOR]

There has been an exponential increase in Photovoltaic energy over the last decade. The size and the complexity of photovoltaic solar power plants are increasing, and it requires advanced and robust condition monitoring systems for ensuring their reliability. To this aim, a novel method is addressed for fault detection in photovoltaic panels through processing of thermal images of solar panels captured by a thermographic camera. In this paper, two advanced convolutional neural network models are used wherein the task of the first model is to classify the type of fault affecting the panel and the task of the second model is to identify the region of interest of the faulty panel. Proposed approach uses F1 score as a metric to compare several classification models of which the ResNet-50 transfer learning model achieves the highest score of 85.37 %. Mean Average Precision is used as an evaluation metric for object detection models wherein the highest scoring model is Faster R-CNN with a score of 67 %. This paper puts forth an approach to facilitate early identification and fault localization in Solar Panels by minimizing the amount of manual labour involved in the process. [ABSTRACT FROM AUTHOR]

As diabetes management becomes more complicated, there is an increasing interest in understanding how to manage diabetes with physical activity. Our study aimed to investigate the role of wearable, non-invasive technologies in collecting data related to physical activity to model them via artificial intelligence methods for efficient diabetes management. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (also known as PRISMA) protocol and searched three databases, namely PubMed, Scopus, and Web of Science. Out of 960 titles, we included 32 in the full-text analysis. Results showed two main methods were used for the analysis, i.e., statistical and classification modeling. Results indicate among the employed regression methods, linear regression was used more than other methods, and the most common classification-based method for analyzing data was the Artificial Neural Network method. Assessing the quality of papers that used the classification method was done through Prediction model Risk Of Bias Assessment Tool (also known as PROBAST) and Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (also known as TRIPOD) tools. Based on PROBAST outcomes, although the risk of bias was low in most of the works, explaining the analyzing method specifically, the method of handling missing data needs more attention. Upon evaluating papers using the TRIPOD, it realized that there is a need to place emphasis on improving the quality of the presentation and explanation of the result. According to our review, the conjunction of non-invasive technologies and artificial intelligence is promising in managing diabetic risk factors for real-time monitoring of physical activities, enabling regular clinical intervention and optimized medical treatment. [ABSTRACT FROM AUTHOR]

In recent years, 3D point clouds (PCs) have gained significant attention due to their diverse applications across various fields, such as computer vision (CV), condition monitoring (CM), virtual reality, robotics, autonomous driving, etc. Deep learning (DL) has proven effective in leveraging 3D PCs to address various challenges encountered in 2D vision. However, applying deep neural networks (DNNs) to process 3D PCs presents unique challenges. This paper provides an in-depth review of recent advancements in DL-based industrial CM using 3D PCs, with a specific focus on defect shape classification and segmentation within industrial applications. Recognizing the crucial role of these aspects in industrial maintenance, the paper offers insightful observations on the strengths and limitations of the reviewed DL-based PC processing methods. This knowledge synthesis aims to contribute to understanding and enhancing CM processes, particularly within the framework of remaining useful life (RUL), in industrial systems. • Survey 3D PC defect classification and segmentation in industrial systems. • Discusses taxonomies categorizing existing models based on data handling designs. • Comparison analysis of the models for industrial damage identification. • Discusses strengths and limitations of DL-based processing methods for 3D PC. • Identifies research gaps and suggests potential avenues for future advancements. [ABSTRACT FROM AUTHOR]

Computer vision-based methods for civil structure's vibration displacement measurement have emerged as useful tools in the recent years. These methods offer several benefits including non-contact measurements, cost-effectiveness, and the ability to capture full-field displacement. Yet, there remain certain challenges. Measuring vibration displacement in 3D typically requires multiple cameras, adding complexity to camera configurations. Moreover, existing methods relied heavily on physical markers or natural key points. Placing physical markers on structures is often impractical, and natural key points are difficult to detect on structures with few distinct features or during rapid movements. Contrary to previous approaches, this paper presents a novel technique that uses a monocular camera for 3D displacement measurements. This technique obviates the need for physical markers or the reliance on natural key points, representing a significant advancement. Central to the method is a deep neural network designed to predict 3D mesh deformation directly from a single image input, combined with an initial 3D cube mesh input. A synthetic 3D dataset is generated to train the neural network. In the testing phase for real structures, advanced video segmentation method is employed to remove the background in order to enhance the prediction accuracy. The practical efficacy of this methodology is validated in a laboratory through a series of experimental tests on beam structures, demonstrating reliable results and application potentials. • This paper proposes a novel monocular vision based 3-dimentional displacement measurement approach for structures. • A stationary monocular camera measurement system is designed to achieve 3-dimensional displacement measurement. • State-of-the-art deep learning approaches are adopted to realize the 3D reconstruction from a single input RGB image. • Two experimental tests are conducted on beam structures in a laboratory to validate the proposed measurement system. [ABSTRACT FROM AUTHOR]

Over the past decades, deep learning techniques have attracted increased attention from various research and industrial domains aligned with the development of Industry Internet-of-Things(IIoT). Specifically, with the advantage of data-driven methods, industrial organizations are seeking novel proactive strategies supported by analytic models to guarantee the quality of their production by observing degradation or predicting failure ahead of the occurrence of the component or asset. Predictive strategies are expected to promise the influence of unnecessary maintenance interruptions and mitigate the consequence of that, hence, extending the remaining useful life of products. This paper conducts a survey of the utilization of deep learning technologies on engineering applications where they provide satisfactory solutions with respect to specific data types or input signals. 106 primary papers are reviewed on deep learning–driven approaches which mainly explore five of the most popular architectures in the application of predictive maintenance. The main content of this paper summarizes the common advantages of each architecture and, accordingly, points out their limitations, as well as describes the application scopes of fully connected deep neural networks, convolutional neural networks, stacked autoencoders, deep belief networks, and deep recurrent neural networks. Based on the technique discussion for each of them, we intend to provide a comprehensive understanding and guidance of the appropriate usage of deep learning architectures to devise an effective predictive maintenance strategy for the scientific and industrial developers whose expertise lies in the prior domain knowledge of multi-source isomerization data. Moreover, the main content demonstrated the summarization of the decisive factor by which the incremental stages of the approaches were determined, fundamentally including the dataset specification, feature extraction, and the integration of deep learning approaches. [ABSTRACT FROM AUTHOR]

• The proposed AP-GAN-DNN method is able to predict creep fracture life more accurately in the case of small samples, which solves the complex nonlinear problem and the problem of test data dispersion. • Compared with traditional physical methods and machine learning models, the AP-GAN-DNN model has better generalization and robustness for creep fracture life prediction. • AP-GAN data enhancement method can significantly reduce the "pattern collapse" effect of GAN and improve the performance of prediction algorithms. 7050 aluminum alloy has high strength and excellent fracture toughness. However, damage can be caused to the 7050 aluminum alloy by creep at elevated temperatures. The accuracy of conventional creep life prediction methods may fall short of meeting the desired standards. And the application of machine learning (ML) methods is limited due to the scarcity and dispersion of creep sample data. Generative Adversarial Networks (GAN) are often used for data enhancement, but suffer from "pattern collapse" under small sample conditions. To solve the above problems, this paper proposes a creep fracture life prediction method, which combines the affinity propagation (AP) clustering algorithm, GAN, and deep neural networks (DNN) for accurate creep fracture life prediction. The AP clustering algorithm for adaptive clustering was utilized to better reflect the creep sample distribution. Independent GAN models were trained for each cluster to better capture distribution characteristics of the data and generate synthetic data that was highly similar to the real data. The DNN models were trained using the synthetic data and then predicted using real creep fracture life data. The presented method was compared with traditional physical and machine learning methods, and the method combining K-Means, GAN, and DNN. The experimental results show that the method proposed in this paper has better prediction accuracy in a small sample creep fracture life dataset. [ABSTRACT FROM AUTHOR]

Deep neural networks have played an important role in the automatic classification of sleep stages due to their strong representation and in-model feature transformation abilities. However, class imbalance and individual heterogeneity which typically exist in raw EEG signals of sleep data can significantly affect the classification performance of any machine learning algorithms. To solve these two problems, this paper develops a generative adversarial network (GAN)-powered ensemble deep learning model, named SleepEGAN, for the imbalanced classification of sleep stages. To alleviate class imbalance, we propose a new GAN (called EGAN) architecture adapted to the features of EEG signals for data augmentation. The generated samples for minority classes are used in the training process. In addition, we design a cost-free ensemble learning strategy to reduce the model estimation variance caused by the heterogeneity between the validation and test sets, to enhance the accuracy and robustness of prediction performance. We show that the proposed method improves classification accuracy compared to several existing state-of-the-art methods. The overall classification accuracy and macro F1-score obtained by our SleepEGAN method on three public sleep datasets are: Sleep-EDF-39: 86.8% and 81.9%; Sleep-EDF-153: 83.8% and 78.7%; SHHS: 88.0% and 82.1%. • This paper develops a new GAN-enhanced ensemble deep learning model, called SleepEGAN, for the imbalanced classification of sleep stages. • It targets to solve two problems commonly existing in sleep or EEG signal-related data — the class imbalance and individual heterogeneity of EEG signals. • It proposes a new EGAN model that can effectively and efficiently generate EEG signals for small classes of sleep stages, and it develops a cost-free ensemble algorithm that can solve the problem of individual heterogeneity of EEG signals. • The proposed method meets the immediate needs of practitioners for seeking an automatic toolbox to classify the sleep stages in the sleep data which are imbalanced and heterogeneous. • It can be conveniently used for automated sleep monitoring and promotes modern wearable technologies. [ABSTRACT FROM AUTHOR]

With the increasing complexity of the power grid, the precision of the thermal power units' execution of automatic generation control (AGC) commands is gradually increasing the impact on the online regulation of the power grid. The deviation between the actual output of thermal power units and the AGC command of the grid will not only affect the consumption of new energy output, but also endanger the safe operation of the grid. This paper introduces "deep learning" technology to solve the problem. Firstly, an AGC command execution effect identification and confidence evaluation algorithm (ACEEI-CEA) is proposed. The algorithm builds a neural network model to accurately predict the unit output and the confidence evaluation of the prediction results. Next, a high-dimensional input preprocessing strategy based on variational autoencoder (VAE) is proposed to reduce the dimensionality of the model input attributes, improving the convergence and accuracy of the model. Finally, an AGC optimal command fast inversion solution method (AOCFISM) is designed. This method transforms the unit output deviation problem into an objective optimisation problem. And improve the efficiency of solving the optimal AGC command value by constraining the unit command value. The calculation results show that the error of the prediction results of the model proposed in this paper is 5% lower than that of the traditional neural network. The difference between the output value of the optimal AGC command and the expected output value obtained is less than 0.5 MV, which can support AGC online decision-making.© 2017 Elsevier Inc. All rights reserved. • An AGC command execution effect identification and confidence evaluation algorithm (ACEEI-CEA) is proposed. • Results show the ACEEI-CEA performs well in command prediction compared to the state-of-the-art methods. • ACEEI-CEA achieves confidence assessment of predicted values. • The pre-processing strategy based on variational autoencoder shortens the training time of the model. • The AGC optimal command fast inversion solution method (AOCFISM) can obtain the AGC optimal command value. [ABSTRACT FROM AUTHOR]

Transformers are Deep Neural Networks (DNN) that utilize a self-attention mechanism to capture contextual relationships within sequential data. Unlike traditional neural networks and variants of Recurrent Neural Networks (RNNs), such as Long Short-Term Memory (LSTM), Transformer models excel at managing long dependencies among input sequence elements and facilitate parallel processing. Consequently, Transformer-based models have garnered significant attention from researchers in the field of artificial intelligence. This is due to their tremendous potential and impressive accomplishments, which extend beyond Natural Language Processing (NLP) tasks to encompass various domains, including Computer Vision (CV), audio and speech processing, healthcare, and the Internet of Things (IoT). Although several survey papers have been published, spotlighting the Transformer's contributions in specific fields, architectural disparities, or performance assessments, there remains a notable absence of a comprehensive survey paper that encompasses its major applications across diverse domains. Therefore, this paper addresses this gap by conducting an extensive survey of proposed Transformer models spanning from 2017 to 2022. Our survey encompasses the identification of the top five application domains for Transformer-based models, namely: NLP, CV, multi-modality, audio and speech processing, and signal processing. We analyze the influence of highly impactful Transformer-based models within these domains and subsequently categorize them according to their respective tasks, employing a novel taxonomy. Our primary objective is to illuminate the existing potential and future prospects of Transformers for researchers who are passionate about this area, thereby contributing to a more comprehensive understanding of this groundbreaking technology. • The paper presents a comprehensive survey on transformers for deep learning tasks. • The paper conducts a thorough analysis on highly effective models in five domains. • The paper classifies the models based on respective tasks using a proposed taxonomy. • The characteristics of the surveyed models are deeply explored and analyzed. • Future directions and challenges for transformer-based models are deciphered. [ABSTRACT FROM AUTHOR]

This paper explores the potential of self-supervised learning as an alternative to supervised learning in the context of geometry-based 3D object retrieval. With the ongoing digitalization of many industries, an exponentially increasing number of 3D objects are processed by retrieval systems. In order to support new shapes, modern deep learning-based retrieval systems require retraining. The dominant paradigm for optimizing neural networks in this field is supervised classification training. Supervised learning requires time-consuming and expensive data annotation. Moreover, training neural networks for classification introduces a bias towards the classes in the training data, which is undesirable for retrieval systems encountering unseen object types in the wild. Through extensive experiments, we make a direct comparison between supervised and self-supervised learning on four datasets from three different domains (household, manufacturing and medical). For object classes seen during training, self-supervised and supervised learning are competitive. For unseen classes, self-supervised learning outperforms supervised learning in many cases. We conclude that self-supervised learning provides a powerful tool for circumventing labeling costs and providing more robust retrieval systems. [Display omitted] • Self-supervised learning is on par with supervised learning for 3D object retrieval. • Self-supervised object retrieval outperforms supervised learning out-of-distribution. • Object alignment is highly impactful on the performance of retrieval systems. • Non-contrastive siamese networks are a practical option in 3D deep learning. [ABSTRACT FROM AUTHOR]

Over the years, Machine Learning models have been successfully employed on neuroimaging data for accurately predicting brain age. Deviations from the healthy brain aging pattern are associated with the accelerated brain aging and brain abnormalities. Hence, efficient and accurate diagnosis techniques are required to elicit accurate brain age estimations. Several contributions have been reported in the past for this purpose, resorting to different data-driven modeling methods. Recently, deep neural networks (also referred to as deep learning) have become prevalent in manifold neuroimaging studies, including brain age estimation. In this review, we offer a comprehensive analysis of the literature related to the adoption of deep learning for brain age estimation with neuroimaging data. We detail and analyze different deep learning architectures used for this application, pausing at research works published to date quantitatively exploring their application. We also examine different brain age estimation frameworks, comparatively exposing their advantages and weaknesses. Finally, the review concludes with an outlook towards future directions that should be followed by prospective studies. The ultimate goal of this paper is to establish a common and informed reference for newcomers and experienced researchers willing to approach brain age estimation by using deep learning models. • We categorize brain age estimation models based on the deep learning architectures. • We classify deep learning architectures based on the slice/voxel-based input data. • We examine the modalities, size of data, performance and the datasets used. • We enumerate several challenges and trace directions that can be explored further. [ABSTRACT FROM AUTHOR]

• The goal of this paper is to create a sketch dataset that is suitable for developing deep learning-based solutions to the problem of search and retrieval in 3D CAD models. • A sketch dataset of query images, called 'CADSketchNet' has been built using the available CAD datasets. For the 58,696 CAD models of the MCB Dataset, we obtain computer-generated sketches. We call this as Dataset-A. We obtain hand-drawn sketch data for all the 801 CAD models from ESB dataset. This is the Dataset-B. • For obtaining Dataset-A, many computer sketch generation methods, including the current state-of-the art methods are experimented. We justify the proposed weighted canny technique based on the comparison. • An analysis of the quality of the hand-drawn sketches in Dataset-B is also performed. • Various deep learning architectures have been tested on the developed CADSketchNet dataset, and a detailed analysis of their performance have been reported. • Among the many experiments conducted, it is shown that a Siamese Network approach yielded the best results as compared to other features/networks. The class-wise results of the approach on both Datasets - A & B. • We also justify the need for a dedicated dataset for CAD over the regular 3D graphical shapes by testing the existing methods on the developed CADSketchNet. [Display omitted] Ongoing advancements in the fields of 3D modelling and digital archiving have led to an outburst in the amount of data stored digitally. Consequently, several retrieval systems have been developed depending on the type of data stored in these databases. However, unlike text data or images, performing a search for 3D models is non-trivial. Among 3D models, retrieving 3D Engineering/CAD models or mechanical components is even more challenging due to the presence of holes, volumetric features, presence of sharp edges etc., which make CAD a domain unto itself. The research work presented in this paper aims at developing a dataset suitable for building a retrieval system for 3D CAD models based on deep learning. 3D CAD models from the available CAD databases are collected, and a dataset of computer-generated sketch data, termed 'CADSketchNet', has been prepared. Additionally, hand-drawn sketches of the components are also added to CADSketchNet. Using the sketch images from this dataset, the paper also aims at evaluating the performance of various retrieval system or a search engine for 3D CAD models that accepts a sketch image as the input query. Many experimental models are constructed and tested on CADSketchNet. These experiments, along with the model architecture, choice of similarity metrics are reported along with the search results. [ABSTRACT FROM AUTHOR]