Search

Deep metric learning aims to construct an embedding space where samples belonging to the same class are closely grouped together, while samples from different classes are widely separated. Many existing deep metric learning methods focus on maximizing the difference between inter-class features, and semantic related information is obtained by increasing the distance between samples of different classes in the embedding space. However, by compressing all positive samples together and creating large margins between different classes, the local structure between similar samples is inadvertently disrupted. This disregard for the intra-class variance presents in the local structure results in an embedding space that exhibits lower generalizability when faced with unseen classes. Consequently, the network tends to overfit the training set and performs poorly on the test set, potentially leading to crashes during evaluation. Taking this into account, this paper introduces a self-supervised generative assisted ranking framework that offers a semi-supervised perspective on learning intra-class variance within traditional supervised deep metric learning. Specifically, this paper employs various intensities and diversities to synthesize samples based on positive examples, aiming to simulate the complex variations between similar samples. Then, an intra-class ranking loss function is devised based on the principles of self-supervised learning. This loss function constrains the ordering relationship of synthesized samples according to their generation intensity, enabling the network to capture subtle intra-class differences and maintain the intra-class distribution. With the adoption of this approach, a more realistic embedding space can be achieved, preserving both the global and local structures of samples. [Display omitted] • We propose a generative boundary that improves the accuracy of generation. • The generative boundary can balance inter-class variance and intra-class variance. • We propose a ranking loss to learn the intra-class variance. [ABSTRACT FROM AUTHOR]

Quick response codes are widely utilized in commodity anti-counterfeiting, and their traceability via smartphones is an effective authentication method. However, quick response codes are susceptible to illegal duplication, and the images captured by smartphones are easily affected by changes in light and environmental noise, leading to unsatisfactory verification results. To address these issues, this paper introduces a novel approach by combining the squeeze-excitation attention module with bottleneck residual block. It presents a squeeze-excitation bottleneck residual network for printer source identification of quick response codes. The squeeze-excitation attention module pays more attention to the features that represent the printer attributes, reduces the interference of useless information, and has low computational consumption; while the bottleneck residual block has the advantages of few parameters, strong feature extraction capability, and good expandability. Thus, the performance can be improved effectively only with a small increasing in parameters. The experimental results verify that the proposed method achieves an accuracy of 98.77% under smartphone capture conditions, it outperforms other convolutional neural network-based methods in terms of identification accuracy. The deep learning model proposed in this paper can be generalized and applied to the printer source identification of paper content in the civil, criminal investigation and judicial fields. [ABSTRACT FROM AUTHOR]

With the increasing complexity of the power grid, the precision of the thermal power units' execution of automatic generation control (AGC) commands is gradually increasing the impact on the online regulation of the power grid. The deviation between the actual output of thermal power units and the AGC command of the grid will not only affect the consumption of new energy output, but also endanger the safe operation of the grid. This paper introduces "deep learning" technology to solve the problem. Firstly, an AGC command execution effect identification and confidence evaluation algorithm (ACEEI-CEA) is proposed. The algorithm builds a neural network model to accurately predict the unit output and the confidence evaluation of the prediction results. Next, a high-dimensional input preprocessing strategy based on variational autoencoder (VAE) is proposed to reduce the dimensionality of the model input attributes, improving the convergence and accuracy of the model. Finally, an AGC optimal command fast inversion solution method (AOCFISM) is designed. This method transforms the unit output deviation problem into an objective optimisation problem. And improve the efficiency of solving the optimal AGC command value by constraining the unit command value. The calculation results show that the error of the prediction results of the model proposed in this paper is 5% lower than that of the traditional neural network. The difference between the output value of the optimal AGC command and the expected output value obtained is less than 0.5 MV, which can support AGC online decision-making.© 2017 Elsevier Inc. All rights reserved. • An AGC command execution effect identification and confidence evaluation algorithm (ACEEI-CEA) is proposed. • Results show the ACEEI-CEA performs well in command prediction compared to the state-of-the-art methods. • ACEEI-CEA achieves confidence assessment of predicted values. • The pre-processing strategy based on variational autoencoder shortens the training time of the model. • The AGC optimal command fast inversion solution method (AOCFISM) can obtain the AGC optimal command value. [ABSTRACT FROM AUTHOR]

The field of comic recognition (CR) has gained significant attention in recent years due to the growing popularity of digital comics. CR involves the automatic identification and extraction of comic panels, speech bubbles, and text from digital comic images, which has numerous applications in the digital media industry. The objective of this review paper is to conduct a systematic literature analysis of the current state of the art in CR techniques and methods. The authors strive to present all major discoveries related to representation models, techniques, tools, datasets, and comparative evaluations of recognition models. The present study utilizes the conventional Systematic Literature Review (SLR) approach and covers a wide range of 72 articles published in 17 prominent journals and 29 notable conferences and workshops. The available literature on recognition models and techniques is broadly categorized into three groups: computer vision (CV) technique being the most commonly used (75%) in the selected studies, followed by non-CV techniques (13.88%) and hybrid techniques (11.11%). Among the widely used datasets in the selected studies, Manga 109 and Ebdtheque are recognized by 50% of them. Within the field of CR, authors such as Jean-Christophe Burie, Christophe Rigaud, Jean-Marc Ogier, and Zhi Tang have received significant attention. The paper summarizes the findings of the CR research and highlights the need for standardization of accuracy norms and datasets. It emphasizes the importance of exploring and developing hybrid techniques for efficient CR tasks. [ABSTRACT FROM AUTHOR]

Industrial products typically lack defects in smart manufacturing systems, which leads to an extremely imbalanced task of recognizing surface defects. With this imbalanced sample distribution, machine learning and deep learning algorithms preferentially learn features from the majority classes, potentially leading to inaccurate results. Addressing the issue of sample imbalance has thus emerged as a critical area of research within the field of industrial intelligent manufacturing. This paper discusses the imbalanced sample problem of industrial product surface defect detection algorithms, and proposes the existence of "four imbalances and two uncertainties". It also summarizes the industrial product surface dataset and innovatively adds the imbalance rate comparison to the dataset. In this study, data re-sampling, data expansion, feature extraction and identification, and re-weighting of category weights are elaborated at the level of data and algorithm respectively. Additionally, the paper explores prospective directions for future research, including supervised and unsupervised learning, transfer learning, anomaly detection, quality prediction, and future challenges. It is hoped to lay a solid foundation for the more far-reaching development of smart manufacturing and surface defect detection methods. And provide some directions for the research of sample imbalance and long-tail problems. [ABSTRACT FROM AUTHOR]

The fault management process is facilitated by equipping power distribution systems with automated devices, especially remote-controlled switches (RCSs). Although RCS plays a key role in improving the system's reliability, it imposes significant investment, installation, and maintenance costs. Hence, RCSs should be optimally located in distribution feeders for the highest profit. In previous works, reliability-oriented mathematical optimization models have been formulated to reach this goal. However, the number of test solutions exponentially grows with the problem size to find the globally optimal solution. This paper uses machine learning to propose a scalable and easy-to-implement model for optimal switch placement in real power distribution systems. At first, the features of candidate points for installing RCSs are introduced. Then, a learning model is applied to deeply explore the relationship between these features and optimal locations for installing RCSs. After training, the learning-based surrogate model directly determines the optimal RCS placement strategy in real power distribution systems by leveraging knowledge gained from past experiences. Simulation results demonstrate that the proposed surrogate model is approximately 29 times faster than the integer programming-based mathematical model, without a significant loss of accuracy, when implemented on a modified 11 kV network connected to Bus 4 of the Roy Billiton test system. This paper also employs explainable artificial intelligence (XAI) tools to select the most important features, where the Hamming loss is decreased by approximately 5%. [ABSTRACT FROM AUTHOR]

Accurate prediction of structural response under earthquake is of great significance for structural damage and performance evaluation. In order to improve the efficiency of structure time history response prediction, this paper proposes a novel SeisFormer model based on the self-attention mechanism and deep learning technology. Through autoregressive prediction, the SeisFormer can achieve real-time prediction of the response time histories of a large number of nodes in the structure under seismic action and can effectively solve the problem of data scarcity. Four case studies are performed to verify the accuracy and efficiency of the proposed methodology, including validation on datasets obtained from elastoplastic seismic analysis of a single-story structure, a three-story structure, and an eleven-story structure, and measured data of a shaking table test model. In addition, this paper further studies the prediction accuracy of the SeisFormer through ablation experiments and comparative experiments. The experimental results show that the SeisFormer can accurately predict the acceleration, velocity, and displacement time histories of numerous nodes in the structure. The prediction accuracy outperforms the LSTM model, and the prediction speed is 193–109,824 times faster than finite element method. Furthermore, with data augmentation through autoregressive prediction, the SeisFormer model can achieve efficient and accurate predictions when training data is exceptionally scarce, enabling engineering applications. • A self-attention mechanism-based method for real-time refined structural response prediction. • An autoregressive data utilizing strategy. • A new deep learning model called SeisFormer. • The SeisFormer outperforms LSTM in structural response prediction under seismic action. • The SeisFormer predicts structural response 193 to 109,824 times faster than the traditional finite element method. [ABSTRACT FROM AUTHOR]

For shafting rotation equipment, in the fault diagnosis based on vibration analysis, the sampling signal is a waveform of a short moment, which is quasi-static information at the corresponding moment. A single quasi-static piece of information does not necessarily contain obvious fault information, however, in a longer period much larger than the sampling period, the time series information composed of multiple quasi-static combinations may well-characterize the fault. Traditional deep learning algorithms often focus on fault features in quasi-static information, ignoring time series features, which results in low diagnostic accuracy. To solve the above problems, this paper firstly presents a fault diagnosis framework of time series and deep fusion network (TDFN). Then, a method to extract time series information and quasi-static information based on expert experience knowledge is proposed, and finally studies the method of using this framework to train the network and fuse the two types of features. Different from existing deep learning methods, TDFN includes time series blocks, depth blocks and fusion blocks. This paper uses laboratory data and industrial rotor data for verification, the results show that the overall accuracy of TDFN reached 93.8% and it does not experience a drop in accuracy after fusion. The diagnosis accuracy is higher than traditional networks, providing guiding suggestions for the management and operation of on-site industrial equipment. [ABSTRACT FROM AUTHOR]

The fundus is the only region where arteries, veins and capillaries can be directly observed. Morphological changes of retinal vessels in the fundus are signals for the appearance of many fundus and cardiovascular diseases. Consequently, the segmentation of retinal vessels is crucial for diagnosing and screening various diseases. In recent years, a large amount of research publications has been published on retinal vessel segmentation. This paper offers a comprehensive review of retinal vessel automatic segmentation research, covering both traditional methods and deep learning methods, including unsupervised and supervised learning methods. Especially, the supervised learning methods are summarized and analyzed from three aspects: traditional CNN-based, GAN-based, and UNet-based methods. This paper also presents an overview of the development of retinal vessel automatic segmentation and analyzes the advantages and disadvantages of existing segmentation methods. The results are shown in two at-a-glance tables. Finally, our work provides a faster and better look to recognize and understand the field of retinal vessel segmentation. • The advancements of automatic retinal vessel segmentation have been summarized. • A summary has been conducted from both supervised and unsupervised learning. • We analyzed the advantages and disadvantages of existing vessel segmentation methods. • Our work provides a more efficient and comprehensive way to understanding the field. [ABSTRACT FROM AUTHOR]

The long-term accumulation of ammonia nitrogen in aquaculture seriously affects the life of fish and even causes large-scale death. Moreover, when the concentration of ammonia nitrogen starts to accumulate, it is a judgment standard to provide early warning through the changes in fish behavior to prevent excessive ammonia nitrogen in water. Therefore, this paper proposes a novel approach to monitoring water quality for aquaculture based on deep learning and three-dimensional movement trajectory. The improved YOLOv8 model was used as the object detection approach to obtain three-dimensional position information of fish by combining Kalman filter, Kuhn Munkres (KM) algorithm, and Kernelized Correlation Filters (KCF) algorithm. The proposed approach was evaluated in the recovery experiment of acute ammonia nitrogen stress of sturgeon, bass, and crucian. The experimental results show that the precision, recall, mAP@0.5 , and mAP@0.5:0.95 of the improved YOLOv8 model are 0.964, 0.914, 0.979, and 0.602, respectively. In addition, the proposed three-dimensional positioning approach can qualitatively and quantitatively analyze the fish behavior in different stages and further explores the fish behavior changes through behavior trajectories, volumes of exercise, spatial distribution, and movement velocity. This research provides a new method and idea for studying the abnormal behavior of aquatic animals under ammonia nitrogen stress and has theoretical and practical significance. • An object detection method combining three-dimensional position information is proposed. • This paper proposes improved YOLOv8 approach combined with multi head self-attention and Wise-IoU. • Quantitative analysis is conducted on the behavioral changes of fish under ammonia nitrogen stress. • This research provides a new method and idea for studying the abnormal behavior of aquatic animals. [ABSTRACT FROM AUTHOR]

The classification rate is the most significant factor affecting the selection of an appropriate classification approach for achieving the desired quality of decisions. Several researchers have evaluated the influence of different features on the performance of classification approaches, but cost/loss functions, despite their theoretical impact on the classification rate, have been less considered in the comparative literature review. In this paper, a comparative study has been presented on the influence of different cost/loss functions on the classification rate of different classifiers. For this purpose, five of the most well-known and widely used categories of cost/loss functions and three of the most popular main categories of classification approaches are considered. Also, 23 benchmark datasets from six different domains are selected. In this paper, all modeling of the statistical and intelligent classifiers are run in the Eviews and MATLAB package software, respectively. In addition, all mathematical modeling of models run in the GAMS package software. Empirical results indicate that cost/loss functions can significantly affect the classification rate. Numerical results show that the discrete cost/loss function, by average 3.91% and 8.31% improvement rather than semi-continuous and continuous cost/loss functions, respectively, is the most effective one. This clearly illustrates that the consistency of the cost/loss function with the goal function of the classification approach has a positive direct relationship with the classification rate. Furthermore, nonlinear versions of semi-continuous and continuous cost/loss functions are on average slightly more efficient than linear ones. Numerical results indicate that nonlinear forms can averagely yield an 88.99% classification rate that is overall 0.98% higher than linear ones, which achieved an 88.13% classification rate. Although the type of cost/loss function can affect the classification rate of different classifiers, the improvement amount varies based on the type of classifier used. In general, the improvement amount in statistical classifiers, i.e., 15.07% is significantly higher than others, and intelligent by 7.86% and deep learning by 5.20% are in the second and third places, respectively. This overall demonstrates that the complexity of the used classifiers has a negative relationship with the improvement of the classification rate in changing cost/loss function. Finally, numerical results indicate that the difference between improvement amounts achieved by varying cost/loss functions, in general, is not significant and is independent of the type of classifier, as well as the type and domain/area of data. [ABSTRACT FROM AUTHOR]

The aging of transformer oil–paper insulation (OPI) contributes significantly to transformer failure and reduced service life. Therefore, reliable and precise evaluation of the aging status of transformer insulation condition is crucial in averting transformer failure. This article proposes a novel deep learning-based aging condition assessment technique of transformers by combining FDS (Frequency domain spectroscopy) and squeeze-and-excitation-enabled convolutional neural network (SE-CNN). An intrinsic product-based transformation using a Gramian angular field is employed to dimensionally expand the 1-D FDS signals acquired from OPI samples into 2D images. Each feature extraction channel in the SE-CNN-based aging diagnosis framework is assigned a specific weight, enabling the model to concentrate on the most relevant features while discarding the rest. Furthermore, the effect of moisture variation on the model is also considered under three different conditions to ensure the consequence of compensation for the effect of moisture on FDS. A comparison with traditional machine learning models has also been made to ensure the viability of the proposed methodology. Additionally, the model is interpreted by creating explanation weight plots of the feature coefficients using a local interpretable model-agnostic explanations (LIME) method in Python. The findings demonstrate that the novel aging diagnosis framework visually assesses the transformer's OPI system with the highest accuracy of 99.12% in the test set. Furthermore, it facilitates the automatic extraction of deep aging features, thus providing better generalization and convergence capability. [ABSTRACT FROM AUTHOR]

Maintenance planning of networked multi-asset systems is a complex problem due to the inherent individual and collective asset constraints and dynamics as well as the size of the system and interdependencies among assets. Although multi-asset systems have been studied numerous times in the past decades, maintenance planning implications of the system's network characteristics have been barely analysed. Likewise, solutions that consider the network perspective suffer from scalability issues as a network-wide observability is assumed. This paper proposes a network maintenance planning approach based on the decomposition of the multi-asset network into fixed-size localised subnetworks. The overall network maintenance plan is produced by aggregating the subnetwork maintenance plans, which are computed independently via a multi-agent deep reinforcement learning (MARL) algorithm. The results are evaluated against a network-wide approach as well as the commonly-used individual approach. The paper also introduces a systematic approach to integrate the MARL resulting policy in a multi-asset agent-based model. Simulation results of several random asset networks and a large nationwide network infrastructure show that, although a network-wide approach outperforms, on average, other approaches considered, the localised subnetworks approach, provides an acceptable alternative in networks with small-world properties, without the need of a network-wide view. • A Network-wide maintenance approach outperforms individual approaches in networked assets with the topologies analysed. • In analysed networks with small-world properties, the Localised approach shows performance close to the network-wide approach. • A systematic integration of MARL and multi-asset agent-based modelling enables evaluation of maintenance approaches. [ABSTRACT FROM AUTHOR]

The accuracy of fault diagnosis is an important indicator to ensure the reliability of key equipment systems. Ensemble learning integrates different weak learning methods to obtain stronger learning and has achieved remarkable results in the field of fault diagnosis. This paper reviews the recent research on ensemble learning from both technical and field application perspectives. The paper summarizes 87 journals in recent web of science and other academic resources, with a total of 209 papers. It summarizes 78 different ensemble learning based fault diagnosis methods, involving 18 public datasets and more than 20 different equipment systems. In detail, the paper summarizes the accuracy rates, fault classification types, fault datasets, used data signals, learners (traditional machine learning or deep learning-based learners), ensemble learning methods (bagging, boosting, stacking and other ensemble models) of these fault diagnosis models. The paper uses accuracy of fault diagnosis as the main evaluation metrics supplemented by generalization and imbalanced data processing ability to evaluate the performance of those ensemble learning methods. The discussion and evaluation of these methods lead to valuable research references in identifying and developing appropriate intelligent fault diagnosis models for various equipment. This paper also discusses and explores the technical challenges, lessons learned from the review and future development directions in the field of ensemble learning based fault diagnosis and intelligent maintenance. [ABSTRACT FROM AUTHOR]

This paper proposes a real-time state of health (SOH) estimation model based on a deep learning (DL) framework. The proposed model is a combination of two different architectures; a one-dimensional convolutional neural network (1D-CNN) and a bidirectional gated recurrent unit (BiGRU). The hybrid CNN-BiGRU uses the 1D CNN layers to extract pertinent features from input data and then relies on the BiGRU layers for sequence learning in both directions. To account for all SOH indicators, the proposed approach uses the current, voltage, and temperature measurements, which are readily obtainable from the electric vehicle's battery management system (BMS). This prevents the complex and time-consuming feature extraction used in most related papers. Since the hyperparameters have a significant impact on the performance of neural network models, a Bayesian Optimization (BO) technique based on the Gaussian Process (GP) was considered to tune the CNN-BiGRU model hyperparameters. Accordingly, the objective function was able to converge to a low Mean Squared Error (MSE) of 1.2 × 10 − 5 in just 19 iterations. Afterward, to verify the accuracy of the optimized model, a Lithium-ion battery dataset with several discharge profiles provided by the National Aeronautics and Space Administration (NASA) was used. The obtained results demonstrated the accuracy and robustness of the proposed method compared to other commonly used models. The CNN-BiGRU model yielded a Mean Absolute Error (MAE) of 2.080% and a root-mean-square error (RMSE) of 2.516% in the case of the battery set #C, referring to a set with 70 cycles already used at 24 °C. Additionally, the End of life (EOL) indicator error of zero cycles for the same data. [ABSTRACT FROM AUTHOR]

Autism spectrum disorder (ASD) is a collection of neuro-developmental disorders associated with social, communicational, and behavioral difficulties. Early detection thereof is necessary to mitigate the adverse effects of this disorder by initiating special education in schools and rehabilitation centers. Two methods are available for diagnosing and rehabilitating ASD. The first is the manual method (i.e., an observation- or interview-based approach), in which the disorder is diagnosed through observation or by interviewing a parent or caregiver. This method is time-consuming, subjective, and mostly comprises the examination of behavioral symptoms. The other method involves automatic diagnosis using traditional machine learning (ML)- and modern deep learning (DL)-based approaches that rely on image analysis. Indeed, the amount of research literature concerned with the evaluation of the usefulness of DL-based methods that process images or video data to diagnose ASD to improve patients' lives has increased significantly. This paper presents a systematic review of the DL-based approach involving the analysis of images or videos in autism research. The review covers studies that were published from 2017 to June 2023 and were indexed in PubMed, IEEE Xplore, ACM Digital Library, and Google Scholar. The results are reported according to the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines. A total of 130 studies were included in the analysis. Eligible papers were categorized based on the different features extracted as input for the DL-based approach. Existing well-known public and private datasets that include images or videos for autism research are extensively reviewed and discussed in this systematic review. In addition, different rehabilitation strategies that have been shown to be highly beneficial for ASD individuals are included. Finally, various challenges presented by the automated detection, classification, and rehabilitation of ASD are discussed. The review concludes that the use of DL for the precise and affordable diagnosis of autism is increasing substantially. Our findings are expected to significantly benefit researchers, therapists, psychologists, and relevant stakeholders to advance ASD screening, monitoring, and diagnosis with the aid of a DL-based approach that entails image or video analysis. [ABSTRACT FROM AUTHOR]

Autonomous-rail Rapid Transit (ART) is a new type of road public transportation with an automatic driving function, and it is one of the main places for epidemic spread. Scientific wearing of masks has been proven to be an extremely effective way to prevent an epidemic. However, drivers wearing masks make it difficult for traditional driver alert detection methods based on facial information to operate effectively. This paper proposes an ART driver alert detection system based on multimodal information fusion and deep learning, which can be applied to drivers wearing masks. The detection system is mainly divided into three modules: data acquisition module, preprocessing module, and vigilance detection module. The data acquisition module collects real-time face thermal imaging data, and environment temperature information of 13 people in the vigilance and fatigue state of the ART simulated driving environment. The preprocessing module proposes a face thermal imaging key information extraction method based on the constraints between Yolov5 and thermal image interframe constraints and extracts the mean temperature information and the hot voxel information (HV) of the eyes, forehead, and mask areas. Then the extracted information and the environment temperature information are transmitted together to the vigilance detection module for learning and classification. The vigilance detection module proposes a lightweight hybrid-attention optimization network, and this paper extracts training sets and testing sets from the data of 13 subjects. And the training set data is enhanced. Each sample is 90 frames. The maximum accuracy rate of the testing set reaches 99.57%. The feasibility of the detection system proposed in this paper is proved, which makes the thermal imaging and attention model show great potential in the application of traffic safety operation and maintenance. • A contactless vigilance detection method is proposed for ART drivers with face masks. • The multi-modal fusion method uses facial thermal image and environmental data. • A lightweight network model that integrates hybrid-attention mechanism is proposed. • A method for extracting face key information from thermal images is proposed. • The testing set shows that the system is feasible and the accuracy rate is 99.57%. [ABSTRACT FROM AUTHOR]

Fruit ripeness estimation models have for decades depended on spectral index features or colour-based features, such as mean, standard deviation, skewness, colour moments, and/or histograms for learning traits of fruit ripeness. Recently, few studies have explored the use of deep learning techniques to extract features from images of fruits with visible ripeness cues. However, the blackberry (Rubus fruticosus) fruit does not show obvious and reliable visible traits of ripeness when mature and therefore poses great difficulty to fruit pickers. The mature blackberry, to the human eye, is black before, during, and post-ripening. To address this problem, this paper proposes a novel multi-input convolutional neural network (CNN) ensemble classifier (MCE) for detecting subtle traits of ripeness in blackberry fruits. The multi-input CNN was created from a pre-trained visual geometry group 16-layer deep convolutional network (VGG16) model trained on the ImageNet dataset. The fully connected layers were optimized for learning traits of ripeness of mature blackberry fruits. The resulting model of about 600 K trainable parameters served as the base for building homogeneous ensemble learners t ensembled using the stack generalization ensemble (SGE) framework. The input to the network are images acquired with a stereo sensor using visible and near-infrared (Vis-NIR) spectral filters at wavelengths of 700 nm and 770 nm. Through experiments, the proposed model achieved 95.1 % accuracy on unseen sets and 90.2 % accuracy with in-field conditions. Further experiments reveal that machine sensory is highly and positively correlated to human sensory over blackberry fruit skin texture • Fruits that do not show obvious and reliable visible traits of ripeness when mature pose great difficulty in fruit ripeness estimation. • Hyperspectral imaging was investigated for evaluating blackberry fruit ripeness and determining the spectral wavelengths that optimally correspond to ripeness in mature blackberry fruit. • A novel multi-input convolutional neural network (CNN) ensemble classifier was proposed for detecting subtle traits of ripeness in images of blackberry fruits. • The proposed model with blackberry fruit images captured at spectral wavelengths of 700 nm and 770 nm achieved an 11.3 % and 16.2 % increase in test accuracies compared to the RGB-based model. • Bi-spectral imagery significantly improves the ripeness-sensing capabilities of deep learning techniques. • Machine sensory has a ripeness sensing capability that is highly and positively correlated to human sensory over blackberry fruit skin texture. [ABSTRACT FROM AUTHOR]

Advances in deep learning and convolutional neural networks (ConvNets) have driven remarkable face recognition (FR) progress recently. However, the black-box nature of modern ConvNet-based face recognition models makes it challenging to interpret their decision-making process, to understand the reasoning behind specific success and failure cases, or to predict their responses to unseen data characteristics. It is, therefore, critical to design mechanisms that explain the inner workings of contemporary FR models and offer insight into their behavior. To address this challenge, we present in this paper a novel template-inversion approach capable of reconstructing high-fidelity face images from the embeddings (templates, feature-space representations) produced by modern FR techniques. Our approach is based on a novel Deep Face Decoder (DFD) trained in a regression setting to visualize the information encoded in the embedding space with the goal of fostering explainability. We utilize the developed DFD model in comprehensive experiments on multiple unconstrained face datasets, namely Visual Geometry Group Face dataset 2 (VGGFace2), Labeled Faces in the Wild (LFW), and Celebrity Faces Attributes Dataset High Quality (CelebA-HQ). Our analysis focuses on the embedding spaces of two distinct face recognition models with backbones based on the Visual Geometry Group 16-layer model (VGG-16) and the 50-layer Residual Network (ResNet-50). The results reveal how information is encoded in the two considered models and how perturbations in image appearance due to rotations, translations, scaling, occlusion, or adversarial attacks, are propagated into the embedding space. Our study offers researchers a deeper comprehension of the underlying mechanisms of ConvNet-based FR models, ultimately promoting advancements in model design and explainability. [Display omitted] Above are example DFD reconstructions of selected sample images from a single subject in the LFW dataset. The rows display: original images (1st row), reconstructions from the VGG–VGG DFD model (2nd row), reconstructions from the VGG–ResNet DFD model (3rd row), reconstructions from the ResNet–VGG DFD model (4th row), and reconstructions from the ResNet–ResNet DFD model (5th row). • We present the Deep Face Decoder (DFD), a novel SOTA template inversion technique. • We use DFD to analyze the embedding space of ConvNet face recognition (FR) models. • Using DFD we explore the impact of image perturbations, occlusions, and attacks. • Through DFD we investigate various template construction strategies. [ABSTRACT FROM AUTHOR]

In the aftermath of natural disasters, efficient waste collection becomes a crucial challenge, owing to the dynamic and unpredictable nature of waste generation, coupled with resource constraints. This paper presents an innovative hybrid methodology that synergizes Long Short-Term Memory (LSTM) machine learning with Differential Evolution (DE) optimisation to augment waste collection efforts post-disaster. The approach leverages real-time data to forecast waste generation with high accuracy, facilitating the development of adaptable waste collection strategies. Our approach is designed to dynamically update collection plans in response to evolving scenarios, ensuring timely and effective decision-making. Field tests conducted in an earthquake-prone city have demonstrated the superior performance of this method in managing waste collection under fluctuating conditions. Moreover, an in-depth sensitivity analysis helps in identifying key areas for improvement. Significantly outperforming traditional models, this method offers substantial time savings and equips disaster response teams with a robust tool for addressing the challenges of waste collection. [ABSTRACT FROM AUTHOR]

Increases in population and prosperity are linked to a worldwide rise in garbage. The "classification" and "recycling" of solid waste is a crucial tactic for dealing with the waste problem. This paper presents a new two-layer intelligent decision system for waste sorting based on fused features of Deep Learning (DL) models as well as a selection of an optimal deep Waste-Sorting Model (WSM) based on Multi-Criteria Decision Making (MCDM). A dataset comprising 1451 samples of images of waste, distributed across four classes – cardboard (403), glass (501), metal (410), and general trash (137), was used for sorting. This study proposes a Multi-Fused Decision Matrix (MFDM) based on identified fusion score level rules, evaluation criteria, and deep fused waste-sorting models. Five fusion rules used in the sorting process and the evaluation perspectives into the MFDM are sum, weighted sum, product, maximum, and minimum rules. Additionally, each of entropy and Visekriterijumska Optimizacija i Kompromisno Resenje in Serbian (VIKOR) methods was used for weighting selected criteria as well as ranking deep WSMs. The highest accuracy rate of 98% was scored by ResNet50-GoogleNet- Inception based on the minimum rule. However, under the same rule, an insufficient accuracy rate of sorting was presented by ResNet50-GoogleNet-Xception. Since Qi = 0 for Inception-Xception, the final output based on MCDM methods indicates that the fused Inception-Xception model outperforms the other fused deep WSMs, which achieved the lowest values of Qi. Thus, Inception-Xception was chosen as the best deep waste-sorting model based on images of waste, multiple evaluation criteria, and different fusion perspectives. The mean and standard deviation metrics were both used to validate the selection findings objectively. The suggested approach can aid urban decision-makers in prioritizing and choosing an Artificial Intelligence (AI)-optimized optimal sorting model. • Develop a 2-layer decision system for waste sorting using deep learning. • Construct MFDM using fusion scores, criteria, and deep waste sorting models. • Entropy and VIKOR methods rank criteria and deep learning models. [ABSTRACT FROM AUTHOR]

Implementing key engineering solutions to optimise the operation of energy industries requires daily electricity demand forecasting and including uncertainty, to promote markets insight analysis as part of their strategic planning, regulating and supplying electricity to consumers. This paper proposes hybrid artificial intelligence models combining convolutional neural networks (CNN) as a feature extraction algorithm with extreme learning machines (ELM) as a framework to predict electricity demand with confidence intervals generated by Kernel Density Estimation (KDE) approaches. In order to develop CELM-KDE model, time-lagged series of daily electricity demand with local climate variables based on the air temperature, atmospheric vapour pressure, evaporation, solar radiation, humidity and sea level pressures are used to train the proposed CELM-KDE hybrid model. In order to fully evaluate the newly developed model from a point-based, as well as a probabilistic prediction strategy, the observed and predicted electricity demand as well as the probability distribution of errors are analysed using KDE method that operates without prior data distribution assumptions. Based on observed and predicted electricity demand and the relevant probabilistic confidence intervals generated by the CELM-KDE model, the final results show that the proposed method attains significantly better probability interval predictions than traditionally-used point-based models. The proposed CELM-KDE model is demonstrated to be highly effective in providing a comprehensive coverage of predicted errors, as well as providing greater insights into the average bandwidth and detailed predicted electricity demand in the testing stage. The results also indicate that the proposed hybrid model is a reliable decision support tool to develop engineering solutions in area of energy modelling, monitoring and forecasting, which could potentially be useful to the industry policymakers. We show that the point-and probabilistic-based electricity demand predictive models can be employed as an effective tool to improve accuracy of forecasting and provision of insights for national electricity markets and key energy industry stakeholder application tools. [Display omitted] • A hybrid deep learning model for daily electricity demand prediction is proposed. • A Kernel Density Estimation is applied to derive probabilistic prediction intervals. • The algorithm quantifies uncertainties in point-based electricity demand predictions. • The proposed model outperforms deep learning models using statistical uncertainty indicators. [ABSTRACT FROM AUTHOR]

Accurately diagnosing the faults of wheelset bearings in high-speed trains is critical for ensuring the safety of train operation. In recent years, deep learning methods have made significant progress in this area. However, collecting enough labeled data in high-speed operation would be highly impracticable. In addition, the harsh operating environment of wheelset bearings, coupled with fluctuations in loads due to passenger flow, weather conditions, and other factors, makes it challenging to identify the fault signatures submerged in background noise. To address these issues, this paper proposes a semi-supervised fault diagnosis method based on autocorrelation and an improved flow Gaussian mixture model. First, the autocorrelations of raw bearing vibration signals are calculated to reduce the impact of noise while highlighting the discriminative fault signatures. Second, an improved flow Gaussian mixture model is established, preserving its simplicity and interpretability, to handle the constraints of bearing fault diagnosis with limited labeled samples under variable loads. Finally, the proposed method is tested by using a bearing dataset collected from an industrial railway wheelset bearing test rig. The results demonstrate that the proposed method outperforms the five state-of-the-art methods in terms of diagnostic accuracy and robustness. • A semi-supervised method is proposed for high-speed train wheelset bearing fault diagnosis with limited labeled samples under variable loads. • Autocorrelation is used to pre-process the raw vibration signals while providing an input to the model. • An improved flow Gaussian mixture model with a simple structure and interpretability is used to diagnose the bearing faults under variable loads. • The effectiveness of the method is verified on a railway wheelset bearing dataset. [ABSTRACT FROM AUTHOR]

Although past learning-based Multi-View Stereo methods performed well, they still struggle to reconstruct regions with occlusions or weak textures. In this paper, we propose a Multi-View Stereo Net using attention mechanism and multi-source cost volume, namely AMS-MVSNet. We first introduce an improved multi-level feature pyramid net (FPN) structure to achieve smoother feature transitions in three stages, and establish additional connections between features that with larger scale difference. This can enhance the fusion of features extracted at different stages. In addition, we construct an attention-enhanced module, which can assign different weights according to the rendering effect of the same spatial point in different views. This can effectively alleviate the impact of false matches caused by weak textures or occlusions during cost volume construction. Furthermore, we utilize a multi-source cost volume that not only incorporates the matching information computed from each view group, but also introduces the depth map differences obtained from different views. The multi-source cost volume greatly enrichs the generalization ability of neural network. Lastly, our network architecture employs a Gated Recurrent Unit (GRU) to reduce memory pressure during the depth inference process and improve efficiency. Our quantitative and qualitative testing results on the DTU, Tanks & Temples and BlendedMVS datasets demonstrate the excellent performance of our neural network. [ABSTRACT FROM AUTHOR]

Accurate wind speed forecasting can improve energy management efficiency and promote the use of renewable energy. However, the inherent nonlinearity and fluctuation of wind speed make prediction challenging. To address these issues, we design an efficient Informer-based model, with improved calculation speed, forecasting accuracy and generalization ability. The proposed model in this paper reasonably integrates the Weibull-Gaussian transform, Informer and kernel mean square error loss and addresses the combination of various components. The Weibull-Gaussian transform is used as the data preprocessing module, which can remove non-Gaussian characteristics from the original data, and thus achieve noise reduction. The Informer is used as the main predictor, which can efficiently output accurate forecasting results based on an encoder-decoder architecture and self-attention mechanism. The kernel mean square error loss function, which shows strong robustness to outliers, is used to evaluate the nonlinearity of errors in reproducing kernel Hilbert space. To evaluate the performance of the proposed model, it is compared with several widely used models and state-of-the-art models. The experimental results indicate that the proposed model weakens the effect of outliers, yields high forecasting accuracy with mean square error = 0.35, and outperforms the baselines up to 8.5% on three datasets. [ABSTRACT FROM AUTHOR]

Unsupervised domain adaptation (UDA) aims to learn robust classifiers for the target domain by leveraging knowledge from annotated source domain. Existing methods concentrated on cross-domain distribution alignment or domain-invariant deep learning model construction. Recently, although the class-specific characteristics have been incorporated for numerous UDA methods, there remains a need for further exploration in capturing intra-class and inter-class characteristics between the source and target domains. In this paper, we propose a novel UDA method, referred to as Class-specific Regularized Joint Distribution Alignment (CRJDA), to simultaneously optimize the intra-class distance of source domain, the inter-class discriminability of target domain, and the joint distribution between domains. Specifically, our method involves an overall optimization process that minimizes the maximum mean discrepancy while incorporating dual penalized class-specific regularizations during joint distribution alignment. Extensive experiments conducted on several benchmark datasets demonstrate that the superiority of the proposed method compared to both state-of-the-art conventional UDA methods and advanced deep UDA models. [ABSTRACT FROM AUTHOR]

Photovoltaic defect detection is an essential aspect of research on building-distributed photovoltaic systems. Existing photovoltaic defect detection models based on deep learning, such as YOLOv5 and YOLOv8, have significantly improved the accuracy of photovoltaic defect detection. However, these models are too large, and their feature extraction ability is insufficient, leading to low detection efficiency and inability to cope with the continuous evolution of defects. Therefore, this study proposes an accurate and lightweight YOLOv8 (You Only Look Once v8) GD algorithm. The algorithm is an improved version of YOLOv8, wherein DW-Conv (DepthWise-Conv) is applied to the YOLOv8 backbone network. Moreover, convolution is replaced with the GSConv (Group-shuffle Conv) and the BiFPN (bidirectional feature pyramid network) structure is added to the architecture. Several electroluminescent photovoltaic defect datasets are used to verify the effectiveness of the proposed method. The final experimental results show that the map@0.5 and map@0.5 ∼ 0.95 of YOLOv8-GD are 92.8% and 63.1%, respectively, which are 4.2% and 5.7% higher than those of the original algorithm, respectively, and the model volume is reduced by 16.7%. Thus, the proposed algorithm shows considerable potential in the field of photovoltaic defect detection. • This paper proposes an improved method based on YOLOv8 for building distributed PV defects, named YOLOv8-GD. • YOLOv8-GD includes improved methods such as GSConv, BiFPN and DW-Conv, which shows better accuracy and speed. • YOLOv8-GD can be used for defect detection and health monitoring in building distributed PV. [ABSTRACT FROM AUTHOR]

Despite massive development in aerial robotics, precise and autonomous landing in various conditions is still challenging. This process is affected by many factors, such as terrain shape, weather conditions, and the presence of obstacles. This paper describes a deep learning-accelerated image processing pipeline for accurate detection and relative pose estimation of the UAV with respect to the landing pad. Moreover, the system provides increased safety and robustness by implementing human presence detection and error estimation for both landing target detection and pose computation. Human presence and landing pad location are performed by estimating the presence probability via segmentation. This is followed by the landing pad keypoints' location regression algorithm, which, in addition to coordinates, provides the uncertainty of presence for each defined landing pad landmark. To perform the aforementioned tasks, a set of lightweight neural network models was selected and evaluated. The resulting measurements of the system's performance and accuracy are presented for each component individually and for the whole processing pipeline. The measurements are performed using onboard embedded UAV hardware and confirm that the method can provide accurate, low-latency feedback information for safe landing support. • An approach to vision-based detection of landing target for autonomous UAV missions. • Near real-time performance of pipeline is confirmed on modern embedded GPU devices. • Increased safety and robustness by human presence detection and error estimation. • Pipeline's accuracy as measured using the OptiTrack system confirms its capability. [ABSTRACT FROM AUTHOR]

3D face reconstruction and dense alignment play a key role in digital humans. Learning-based approaches have been developed by employing cascaded convolutional neural networks for feature extraction and spatial-semantic relationship construction. However, these networks often struggle with accurately capturing the complex relationships among various facial components and attributes, particularly in unconstrained environments. Furthermore, the mappings modeled by convolutional weights are at low levels, which are usually implicit and local, and lack global discrimination. In this paper, we propose a global and instance-level perceived relationship matrices-based network (PRMNet) to recover high-fidelity 3D faces and perform accurate dense alignment in unconstrained environments. Specifically, a Key Information Extraction Module (KIEM) extracts crucial features from global and local feature banks. This reduces reasoning costs while maintaining high perceptual quality. Learnable global and instance-level perceived feature relationship matrices are then integrated into the Feature Relationship Reasoning Module (FRRM). This calibration combines key information from both the global macroscopic and sample-specific microscopic views, allowing accurate construction of spatial-semantic relationships to harvest both global discrimination and local relevance. Finally, we introduce a Spatial Reasoning and Guidance Module (SRGM), designed to recalibrate the joint weight responses of feature extraction and its enhancement paths using various attention mechanisms, thereby further enhancing global discrimination ability. Extensive quantitative and qualitative experiments on the benchmark datasets show that our PRMNet outperforms the state-of-the-art. Codes and all resources will be publicly available at https://github.com/Ray-tju/PRMNet. [ABSTRACT FROM AUTHOR]

Human motion analysis plays a crucial role in industry 4.0 and, more recently, in industry 5.0 where human-centered applications are becoming increasingly important, demonstrating its potential for enhancing safety, ergonomics and productivity. Considering this opportunity, an increasing number of studies are proposing works on the analysis of human motion in an industrial context, taking advantage of the rise of artificial intelligence technologies and sensor technologies. The objective of this work is to provide a review of recent studies exploring these technologies in the analysis of human movement while specifically considering industrial context. First, a taxonomy of key human motion analysis applications is proposed, presenting statistical insights to reveal trends and highlighting lacks in current research. Furthermore, this work identifies benchmark datasets acquired in various industrial case studies and associated sensors. Many recommendations for selecting optimal sensors and valuable benchmarks are proposed. Then, the paper outlines the current trend of utilizing hybrid deep learning methodologies in human movement analysis while underscoring the performance and limitations of these proposed methods, considering industrial constraints such as real-time recognition and frugality. Finally, challenges and future works are highlighted, focusing on the opportunities to address problems related to the complex industrial environment in order to achieve reliable performances. [ABSTRACT FROM AUTHOR]

The global generation of renewable energy has rapidly increased, primarily due to the installation of large-scale renewable energy power plants. However, monitoring renewable energy assets in these large plants remains challenging due to environmental factors that could result in reduced power generation, malfunctioning, and degradation of asset life. Therefore, the detection of surface defects on renewable energy assets is crucial for maintaining the performance and efficiency of these plants. This paper proposes an innovative detection framework to achieve an economical surface monitoring system for renewable energy assets. High-resolution images of the assets are captured regularly and inspected to identify surface or structural damages on solar panels and wind turbine blades. Vision transformer (ViT), one of the latest attention-based deep learning (DL) models in computer vision, is proposed in this work to classify surface defects. The ViT model outperforms other DL models, including MobileNet, VGG16, Xception, EfficientNetB7, and ResNet50, achieving high accuracy scores above 97% for both wind and solar plant assets. From the results, our proposed model demonstrates its potential for monitoring and detecting damages in renewable energy assets for efficient and reliable operation of renewable power plants. • For the first, vision transformer-based model is proposed for electrical asset management. • Framework for autonomous inspection of large-scale renewable power plant's asset damages. • Low cost with less human intervention and less processing time. • Model achieves high accuracy >97% for damage detection on solar panels and wind blades. • ViT outperforms in classification compared to other deep learning models. [ABSTRACT FROM AUTHOR]

Distributed point source method (DPSM) is a collocation point-based semi-analytical method to model the scattered ultrasonic fields in isotropic and anisotropic materials. It requires the evaluation of Green's function solutions and its rigorous differentiation for complex geometry problems having a large set of distributed point sources. DPSM is much faster than the finite element method (FEM) however, it is analytically demanding as it requires differentiation of the Green's function. The intricacy associated with differentiation operation hinders the potential application of DPSM in large-scale automation and real-time structural health monitoring. The current paper introduces machine-learning-based strategies within DPSM and yields the proposed Neural Green-DPSM (NG-DPSM) framework. A physics-informed neural network (PINN) is incorporated in the DPSM framework for evaluating the Green's function and its gradients. To train the PINN, displacement Green's function solutions of wave propagation in isotropic solid is used as the governing physics. Once the PINN is trained for displacement Green's function solutions, the NG-DPSM framework leverages the trained network and its automatic differentiation capability to predict displacement and stress fields annihilating the rigorous differentiation of Green's functions. The accuracy and efficacy of NG-DPSM are demonstrated using two numerical experiments. First, the ultrasonic fields are evaluated for the problem geometry of a plate immersed in fluid excited at different angles of incidence. Further, the efficacy of the proposed approach is demonstrated for wave scattering through a circular hole in the plate. The results show a strong agreement between the ultrasonic fields computed using both NG-DPSM and conventional DPSM. [ABSTRACT FROM AUTHOR]

The purpose of video process detection is to identify key processes of interest and associated semantic categories, which is an important computer vision task for localizing key processes in space science experiments. To detect experimental processes in space electrostatic suspension material experiment on China's Space Station, we propose a Slow-and-Fast Dual-Stream Network. It extracts multi-scale temporal semantic information by designing a Slow-and-Fast Temporal Stream structure and effectively detects the experimental processes with various temporal scales. In addition, the Two Path Temporal Pyramid Network (TP2N) enhances the bi-direction flow among multi-scale features on the temporal dimension to alleviate boundary sensitivity. The validation experiments on our established dataset demonstrate the effectiveness of our proposed method with 62.43% mAP, outperforming the related state-of-the-art temporal action localization methods with nearly 7% mAP gain. This paper is the first application of deep learning in space electrostatic suspension material experiments on China's Space Station and further provides important technical support for the identification of key processes in space science experiments. [ABSTRACT FROM AUTHOR]

Conventional malware detection approaches have the overhead of feature extraction, the requirement of domain experts, and are time-consuming and resource-intensive. Learning-based approaches are the mainstay of malware detection as they overcome most of these challenges by significantly improving the detection effectiveness and providing a low false positive rate. The exponential growth of malware variants and first-time-appeared malware, which includes polymorphic and zero-day attacks, are some of the significant challenges to learning-based malware detectors. These challenges have catastrophic impacts on the detection effectiveness of these learning-based malware detectors. This paper proposes a novel deep learning-based framework to detect first-time-appeared malware effectively and efficiently by providing better performance than conventional malware detection approaches. First, it translates and visualises each Windows portable executable (PE) file into a coloured image to eliminate the overhead of feature extraction and the need for domain experts to analyse the features. In the subsequent step, a fine-tuned deep learning model is used to extract the deep features from the last fully connected layer. The step has reduced the cost of training required by the deep learning models if used for end-to-end classification. The third step selects the most important and influential features through a powerful feature selection algorithm. The most important features are then fed to a one-class classifier for final detection. With the one-class classifier, an enclosed boundary around the features of benign data is constructed. Anything outside the boundary is declared as an anomaly/malicious. It has enhanced the framework's ability to detect evolving, unseen, polymorphic, and zero-day attacks, as well as reducing the problem of overfitting. The detection effectiveness of the proposed framework is validated with state-of-the-art deep learning models and conventional approaches. The proposed framework has outperformed with an accuracy of 99.30% on the Malimg dataset. The Wilcoxon signed-rank test is used to validate the statistical significance of the proposed framework. It is evident from the results that the proposed framework is effective and can be used in the defence industry, resulting in more powerful and robust solutions against zero-day and polymorphic attacks. [Display omitted] • A novel approach of combining deep learning and machine learning is proposed. First, deep learning is used to extract deep features. The most influential and meticulous features are selected in the subsequent steps to train the machine learning classifier for final detection. The proposed framework eliminates the need for human efforts for reverse engineering tasks. • The proposed framework consists of four steps. In the first step, all PEs are transformed into coloured images. The second step used a deep learning model to extract the deep features. The subsequent step selects the most important features. Finally, the lightweight and most influential features are sent to the final machine learning classifier for final malware detection. • We demonstrate that the proposed framework is lightweight, resilient, efficient and cost-effective. An in-depth analysis is performed to validate the detection effectiveness and generalisation of the proposed framework on multiple datasets. Our results demonstrate that the proposed framework outperformed conventional and state-of-the-art malware detection approaches. [ABSTRACT FROM AUTHOR]

Typing is a time-consuming task and predictive text is proposed as a solution. Recently, Generative Pre-trained Transformers (GPT) have employed autoregressive deep learning to tackle text prediction. However, they face costly retraining, especially for low-resource languages (such as Persian) or domains, and lack controllability. Text augmentation with prompting methods and fine-tuning GPT models on templated data using conditional elements (labels or keywords) aims to address these problems, enhancing controllable text generation in low-resource scenarios. These methods involve discovering and mapping conditional elements to training texts, which is unsuitable for typing assistants. Meanwhile, they do hold inappropriate pre-defined elements for wide use. This paper introduces Conditional GPT-2-Persian (CGPT-2-Persian), which utilizes the initial word of each sentence as the associated conditional element to address practical challenges, extracting labels, and handling in-domain unseen data posed by the mentioned methods. This method outperforms Persian text generation methods in terms of BLEU and ROUGE scores, achieving 87.39% and 14.29%, respectively, after fine-tuning for ten epochs. This study can be efficient for other low-resource languages or domains. [ABSTRACT FROM AUTHOR]

Multi-view stereo (MVS) reconstruction is a key task of image-based 3D reconstruction, and deep learning-based methods can achieve better results than traditional algorithms. However, most of the current deep learning-based MVS methods use convolutional neural networks (CNNs) to extract image features, which cannot achieve the aggregation of long-distance context information and capture robust global information. In addition, in the process of fusing depth maps into point clouds, the confidence filters will filter out the depth values with low confidence in weak texture areas. These problems will lead to the low completeness of 3D reconstruction of weak texture and texture-less areas. To address the above problems, this paper proposes SA-MVSNet based on the PatchmatchNet with a self-attentive mechanism. First, we design a coarse-to-fine network framework to advance depth map estimation. In the feature extraction network, a module with a pyramid structure based on Swin Transformer Block is used to replace the original Feature Pyramid Network (FPN), and the self-correlation between weak texture areas is enhanced by applying a global self-attention mechanism. Then, we also propose a self-attention-based adaptive propagation module (SA-AP), which applies a self-attention calculation within depth value propagation window to obtain the relative weight values of current pixel and others, and then adaptively samples the depth values of neighbors on the same surface for propagation. Experiments show that SA-MVSNet has significantly improved the completeness of 3D reconstruction for the images with weak texture on DTU (provided by Danish Technical University), BlendedMVS, and Tanks and Temple datasets. Our code is available at https://github.com/miaowang525/SA-MVSNet. • SA-MVSNet proposed a multi-scale feature extraction module based on Swin Transformer. • A global self-attention mechanism is applied to enhance the self-correlation between weak textured areas. • SA-MVSNet proposed a self-attention-based adaptive propagation module (SA-AP). • Apply window self-attention mechanism to enhance the confidence of depth value. • SA-MVSNet has significantly improved the completeness of 3D reconstruction on three datasets. [ABSTRACT FROM AUTHOR]

This paper addresses the problem for fine-grained video super-resolution (FGVSR) to suppress temporal flickering caused by separately processed consecutive frames and enhance the quality of restored video frame details when upsizing videos. Some existing video SR methods fail to sufficiently utilize spatial-temporal information from input low-resolution (LR) videos, while others may generate undesirable artifacts or cannot well reconstruct image details. To overcome these problems, we present a novel deep learning framework for FGVSR, which takes a set of consecutive LR video frames and generate the corresponding super-resolved frames. Our deep FGVSR framework focuses on reconstructing missing information from the LR sources based on the proposed multi-frame alignment and refinement strategies. More specifically, we propose an alignment module, where multiple frames are aligned at feature level, to prevent the output videos from flickering. Then, we introduce a feature fusion module, where aligned features generated from our alignment module are fused and refined in a multi-scale manner. Finally, the proposed refinement module is used to reconstruct missing information based on the fused features. In addition, we also embed an image enhancement module on the skip connection from the input layer to the output layer of our network for further enhancing the SR results. Experimental results show that the proposed deep FGVSR, compared with existing deep learning-based VSR methods, achieves state-of-the-art performances on the three well-known benchmarks, including REDS, Vid4, and Vimeo90k. More specifically, compared with the state-of-the-art VSR methods in our experiments, our FGVSR achieves quantitative improvements from 0.70 dB to 9.54 dB in PSNR. On the other hand, our method has also been shown to be efficient to other image restoration tasks, such as image inpainting. • End-to-end trainable deep learning-based fine grained video super-resolution model is proposed. • The temporal dependency is implicitly obtained without performing explicit motion compensation. • Image details are reconstructed and further sharpened by our refinement and enhancement modules. • Adaptively integrated into other deep image restoration model for further boosting performances. [ABSTRACT FROM AUTHOR]

In recent years cryptographic tokens have gained popularity as they can be used as a form of emerging alternative financing and as a means of building platforms. The token markets innovate quickly through technology and decentralization, and they are constantly changing, and they have a high risk. Negotiation strategies must therefore be suited to these new circumstances. The genetic algorithm offers a very appropriate approach to resolving these complex issues. However, very little is known about genetic algorithm methods in cryptographic tokens. Accordingly, this paper presents a case study of the simulation of Fan Tokens trading by implementing selected best trading rule sets by a genetic algorithm that simulates a negotiation system through the Monte Carlo method. We have applied Adaptive Boosting and Genetic Algorithms, Deep Learning Neural Network-Genetic Algorithms, Adaptive Genetic Algorithms with Fuzzy Logic, and Quantum Genetic Algorithm techniques. The period selected is from December 1, 2021 to August 25, 2022, and we have used data from the Fan Tokens of Paris Saint-Germain, Manchester City, and Barcelona , leaders in the market. Our results conclude that the Hybrid and Quantum Genetic algorithm display a good execution during the training and testing period. Our study has a major impact on the current decentralized markets and future business opportunities. [ABSTRACT FROM AUTHOR]

In wire arc additive manufacturing (WAAM), the electric arc is an essential part of the welding equipment, which serves as the heat source and is directed by the current and voltage. The working status of the electric arc is an important factor in determining the quality of the fabricated components. During the welding process, the current and voltage may change abruptly due to some abnormalities in the operating conditions, which may affect the working status and thereby affect the quality of products. Such abnormal changes in the current and voltage can be treated as outliers. In order to identify outliers in current and voltage to further improve the welding process, in this paper, a novel deep-transfer-learning-embedded outlier detection approach is developed for WAAM. A new domain adaptation strategy is designed where the cross-domain discrepancies of the marginal distribution and conditional distribution are minimized. Specifically, two separate coefficients are introduced to adjust the conditional domain discrepancies of normal instances and outliers with the purpose of alleviating the data imbalance problem. The particle swarm optimizer is employed to adjust the hyper-parameters. The developed deep transfer learning framework is exploited in designing a new outlier detector with application to WAAM. The proposed approach is exploited in real-world industrial data collected through the WAAM process. Experimental results demonstrate that the proposed outlier detection approach outperforms the standard deep-learning-based outlier detector approach and the standard transfer-learning-embedded outlier detection approach in terms of detection accuracy. [ABSTRACT FROM AUTHOR]

Conventionally, the extraction of hand-crafted acoustic features has been separated from the task of establishing robust machine-learning models in speech processing. The manual approach of feature engineering is both time-consuming and necessitates specialist knowledge, posing significant hindrances. Moreover, the resulting features may not be ideal for the desired application. The speech community has adopted raw waveform modeling to enhance performance. These techniques learn an optimized representation of the input automatically. With deep learning (DL) advancements, raw waveform modeling has become valuable for tasks like classification and prediction. The primary aim of this survey is to offer valuable insights and fills a gap in the existing literature by providing a comprehensive review of the state-of-the-art in speech and speaker recognition using raw waveform modeling for both adult and children's speech. The article covers papers from 2013–2023 and is the first to review both adult and children's databases. The article focuses on the advantages of raw waveform models. It presents essential concepts and techniques while discussing the challenges and limitations of using raw waveforms for speech and speaker recognition in both adult and children's speech. The article also evaluates recent progress in DL architectures such as SincNet, ResNet, and RawNet, and outlines future research directions in the field. [ABSTRACT FROM AUTHOR]

In the realm of low-light image enhancement, the proliferation of deep learning techniques in recent years has led to significant improvements in image quality and clarity. Despite the progress, the conventional use of 3-Dimensional Look-Up Tables (3D LUTs) for color space discretization primarily yields a global average enhancement, which may not cater to the localized semantic nuances of images, and not only neglect the brightness details but also tends to lead to color inconsistencies. In order to solve the above problems, in this paper, our study presents an innovative semantic compensation model built on 3D LUT. The primary objective of this model is to meticulously preserve the color fidelity across various semantic regions, concurrently ensuring the retention of brightness and texture nuances inherent in images. Specifically, we introduce the Semantic Embedding Module (SEM), which meticulously infuses semantic details into the enhancement process, ensuring that the colors of the enhanced images remain true to the original content. In SEM, Feature Aggregation Block (FAB) is proposed to solve the entanglement problem of multi-level features and dynamically adjust the weights, thus helping to reconstruct images with sufficient brightness, complete details, and more realistic color representation. Furthermore, recognizing the dependency of existing methods on semantic labels for acquiring semantic maps, we propose the innovative Semantic Contrast loss. This loss function is instrumental in generating semantic maps independently of semantic labels, thus expanding their practical utility. Our contributions effectively bridge the gap between global enhancement techniques and the need for localized semantic accuracy, paving the way for more sophisticated low-light image enhancement methodologies. Our experimental evaluations, conducted on several synthetic and natural datasets, corroborate the superior performance of our method compared to state-of-the-art methods. We release our code at https://github.com/zyy317077/SAE/. [Display omitted] • The model introduces a semantic compensation mechanism for 3D LUTs of low-light images. • A novel Semantic Contrast loss function captures semantic details without labels. • The Semantic Embedding Module ensures color correction aligns with target content hues. • The Feature Aggregation Block dynamically adjusts features to enhance color correctness while maintaining brightness and detail. [ABSTRACT FROM AUTHOR]

Scene text detection has always been a research hotspot in computer vision and image understanding. With the development of deep learning, segmentation-based methods have achieved an exceptional effect in regular or curved text detection, but they cannot separate adjacent word-level texts. In this paper, we proposed a detector called Buffer-Text for the detection of irregular text in the natural scene image. First, the buffer region is proposed for bending text detection which widens the spatial distance between word-level texts. Then, a centerline-based polygon expansion algorithm is developed for the acquisition of the buffer region. After that, the scene text image is divided into different regions which are predicted by adopting the idea of multiclass semantic segmentation. To obtain effective segmentation results and solve the category imbalance problem, a Fully Convolutional Networks (FCN) with Spatial and Channel Squeeze & Excitation Block module is designed, and a loss function with adaptive weight updating is defined for the network. Ultimately, the post-processing including the total erosion and the single expansion is applied to eliminate the areas of noise in the segmented image and to separate the weak junctions in the word-level text. To verify the validity of the proposed method, several experiments were conducted on two curved text datasets, namely Total-Text and CTW1500, and the results indicated that the proposed method achieved significant accuracy in three statistical indicators (precision, recall, and F-score), particularly for the images with natural scenes and various text shapes. • The introduction of a buffer region aids in curved text detection, extending spatial separation between word-level texts. • A centerline-based polygon expansion algorithm is developed for the acquisition of the buffer region. • A FCN with a self-applicable loss function is crafted to acquire a segmentation map of the text region. • Full erosion followed by a single extended post-processing is recommended to eliminate noise region. [ABSTRACT FROM AUTHOR]

Currently, the fusion of surface Electromyography (EMG) and deep learning is gradually showing immense potential in the research of Lower Limb Motion Intention Recognition (LLMIR). Nevertheless, most deep learning algorithms have poor interpretability without special design or the help of other post-hoc analysis tools, as well as unsatisfactory performance in cross-subject prediction. Hence, this paper presents a novel Interpretable Dual-Branch EMG Network (IDB-EMGNet), in which one branch is dedicated to lower limb motion recognition, and the other is able to predict knee joint angles in advance. The shallow feature extraction module of IDB-EMGNet is constructed using an ante-hoc interpretable SincNet technique, which enables the detection of the spectral range of EMG used for the LLMIR task. An improved bottleneck block with shuffle attention is designed for deep feature extraction, which enhances model performance with only a little increase in complexity. The performance of IDB-EMGNet in both intra-subject and inter-subject scenarios is investigated, where the latter integrates the transfer learning technique. Specifically, by conducting model pre-training on source-domain subjects and transferring the learned knowledge to target-domain subjects, satisfactory performance can be achieved even with less computing resource. Experimental results on two publicly available datasets indicate that the proposed approach exhibits superior applicability to both normal subjects and knee-pathology patients, showing a promising prospect in the controller design of human-robot collaborative exoskeletons. • A novel IDB-EMGNet is proposed for simultaneous discrete and continuous lower limb motion intention recognition. • Interpretability of IDB-EMGNet is enhanced via an ante-hoc SincConv technique. • An IB-Neck block is designed to improve model prediction performance. • Transfer learning technique is utilized to improve model generalization in inter-subject prediction. • The proposed approach performs well for both normal subjects and knee-pathology patients. [ABSTRACT FROM AUTHOR]

Deep learning methods have achieved a lot of success in various applications involving converting wearable sensor data to actionable health insights. A common application areas is activity recognition, where deep-learning methods still suffer from limitations such as sensitivity to signal quality, sensor characteristic variations, and variability between subjects. To mitigate these issues, robust features obtained by topological data analysis (TDA) have been suggested as a potential solution. However, there are two significant obstacles to using topological features in deep learning: (1) large computational load to extract topological features using TDA, and (2) different signal representations obtained from deep learning and TDA which makes fusion difficult. In this paper, to enable integration of the strengths of topological methods in deep-learning for time-series data, we propose to use two teacher networks — one trained on the raw time-series data, and another trained on persistence images generated by TDA methods. These two teachers are jointly used to distill a single student model, which utilizes only the raw time-series data at test-time. This approach addresses both issues. The use of KD with multiple teachers utilizes complementary information, and results in a compact model with strong supervisory features and an integrated richer representation. To assimilate desirable information from different modalities, we design new constraints, including orthogonality imposed on feature correlation maps for improving feature expressiveness and allowing the student to easily learn from the teacher. Also, we apply an annealing strategy in KD for fast saturation and better accommodation from different features, while the knowledge gap between the teachers and student is reduced. Finally, a robust student model is distilled, which can at test-time uses only the time-series data as an input, while implicitly preserving topological features. The experimental results demonstrate the effectiveness of the proposed method on wearable sensor data. The proposed method shows 71.74% in classification accuracy on GENEActiv with WRN16-1 (1D CNNs) student, which outperforms baselines and takes much less processing time (less than 17 sec) than teachers on 6k testing samples. [ABSTRACT FROM AUTHOR]

In this paper, we propose a lightweight approach for industrial defect detection that is modeled based on anomaly detection using point pattern data. Most recent works use global features for feature extraction to summarize image content. However, global features are not robust against lighting and viewpoint changes and do not describe images' geometrical information which, thus, cannot be fully utilized in manufacturing industry. To the best of our knowledge, we are the first to propose using transfer learning of local/point pattern features to overcome these limitations and capture geometrical information of the interested image regions. We model these local/point pattern features as a random finite set (RFS). In addition, we propose RFS energy, in contrast to RFS likelihood as an anomaly score. The similarity distribution of point pattern features of the normal sample has been modeled as a multivariate Gaussian. Parameters learning of the proposed RFS energy does not require any heavy computation. We evaluate the proposed approach on the MVTec AD dataset, a multi-object defect detection dataset. Experimental results show the outstanding performance of our proposed approach compared to the state-of-the-art methods, and the proposed RFS energy outperforms the state-of-the-art in the few shot learning settings. Also, we evaluated the proposed approach on needle dataset, a real life application, to detect defective needle samples given three different views. The results show superior performance compared to DiffeNet. [Display omitted] • A lightweight approach for defect detection based on anomaly detection is proposed. • Transfer learning of deep local features is used instead of global features. • We model these local/point pattern features as a random finite set (RFS). • We propose RFS energy, in contrast to RFS likelihood as an anomaly score. • Results show the outstanding performance of our proposed approach compared to the state-of-the-art methods in few-shot learning. [ABSTRACT FROM AUTHOR]

The aim of this paper is to investigate emotion recognition using a multimodal approach that exploits convolutional neural networks (CNNs) with multiple input. Multimodal approaches allow different modalities to cooperate in order to achieve generally better performances because different features are extracted from different pieces of information. In this work, the facial frames, the optical flow computed from consecutive facial frames, and the Mel Spectrograms (from the word melody) are extracted from videos and combined together in different ways to understand which modality combination works better. Several experiments are run on the models by first considering one modality at a time so that good accuracy results are found on each modality. Afterward, the models are concatenated to create a final model that allows multiple inputs. For the experiments the datasets used are BAUM-1 ((Bahçeşehir University Multimodal Affective Database - 1) and RAVDESS (Ryerson Audio–Visual Database of Emotional Speech and Song), which both collect two distinguished sets of videos based on the different intensity of the expression, that is acted/strong or spontaneous/normal, providing the representations of the following emotional states that will be taken into consideration: angry, disgust, fearful, happy and sad. The performances of the proposed models are shown through accuracy results and some confusion matrices, demonstrating better accuracy than the compared proposals in the literature. The best accuracy achieved on BAUM-1 dataset is about 95%, while on RAVDESS it is about 95.5%. [Display omitted] • Emotion recognition through multimodal architectures. • Comparison of early to late fusion of 1-input models. • Synchronization of video and audio channels. [ABSTRACT FROM AUTHOR]

Understanding thermal dynamics and obtaining the computational model of residential buildings enable its scaled application in energy retrofits, control optimization and decarbonization. In this paper, we present a deep learning approach to model building thermal dynamics with smart thermostat data collected from residential buildings, with the goal to investigate model generalizability. In the first stage, we developed and compared different Deep Learning architectures including Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM) models and CNN-LSTM to predict indoor air temperature in a multi-step time horizon. In the second stage, we implemented a Transfer Learning (TL) process, which aims to improve the prediction performance on a new set of buildings (targets), exploiting the knowledge of related or similar buildings (sources). Different TL strategies and source model identification methods were investigated. The study showed that the CNN-LSTM performed the best among the architectures compared, with an average Mean Absolute Error (MAE) of 0.26 °C for one-hour-ahead (twelve 5-min future steps) predictions. Furthermore, the results showed that freezing the LSTM layer and fine-tuning the other layers of the CNN-LSTM achieved the best performance among four TL strategies, which further improved the performance with respect to a machine learning approach by 10%, and proving the effectiveness and generalizability of the proposed approach. A comparison of three different source model identification methods showed that randomly selecting source models constrained by similar building characteristics can provide good TL performance while retaining simplicity comparing with other quantitative source identification methods. • Investigated deep transfer learning techniques for residential thermal dynamics using smart thermostat data. • Compared diverse neural network architectures for indoor temperature prediction. • Explored transfer learning to enhance model accuracy. • Evaluated four transfer learning methods and three source model identification approaches. • Achieved high accuracy and generalizability with the proposed deep transfer learning approach. [ABSTRACT FROM AUTHOR]

Predicting the remaining useful lifetime (RUL) of bearings in electric motors is crucial to reduce repair costs in industrial maintenance. With the technological advances of Industry 4.0, physical models for prognostics and RUL prediction have been replaced by data-driven models that require no expert feature extraction. Instead, the model itself learns which features are important. However, these models are normally trained and tested on the same dataset, i.e. under the same operating conditions. This limits the application of a model to other operating conditions unless the model is fine-tuned with data corresponding to those conditions. This paper proposes a novel robust health prognostics technique that detects inner-race bearing failures and predicts the RUL of electric motor bearings under various motor conditions without model retraining or fine-tuning. The model combines time and frequency-domain vibration signal analyses to extract features, a stacked variational denoising autoencoder (SVDAE) to fuse these features and build a Health Indicator and a bidirectional long short-term memory (BiLSTM) neural network to predict the remaining useful lifetime of the bearings. The proposed model is trained with a dataset, validated with another dataset and finally tested with seven additional datasets corresponding to vibrations gathered from different motors and operating conditions. The results are more robust and accurate than those of the literature, the robustness of the prediction with different motor and operating conditions is proven, and there is no need to retrain or fine-tune the model, making the proposed model suitable for recently installed equipment. • Inner-race bearing frequencies are used to predict RUL of electric motor bearings. • SVDAE and BiLSTM help the proposed prediction model improve the literature results. • Random search optimization technique is used for hyperparameter selection of BiLSTM. • Robustness while predicitng is achieved under different motor operating conditions. • Proposed solution can determine the failure apart from predicting the RUL. [ABSTRACT FROM AUTHOR]

The lattice structures obtained by combining repetitive light cellular forms provide superior high strength to weight capabilities compared to monolithic solid bodies. These properties of lattice structures can be further enhanced by improving the design and manufacturing process. However, the process of creating aluminum alloy lattice structures with tailored mechanical properties is still challenging due to the wide range of possible designs and their complex structure-property relationships. This paper proposes a new methodology that uses the deep learning-based 3D Generative Adversarial Network (3DGAN) model to solve this complex engineering problem. Unlike previous studies on deep learning-based lattice design, the dataset obtained through parametric design and Simulated Annealing techniques enables GAN to create new 3D lattice structures with improved mechanical strength properties. The designs generated with the GAN algorithm were produced using Selective Laser Melting Additive Manufacturing (SLM-AM) technology. The mechanical properties of SLM-fabricated (SLMed) AlSi10Mg unit cell samples were examined by a series of compression and impact tests. The results reveal that the lattice configurations generated using the generative model exhibit improved mechanical properties (e.g., a remarkable increase in normalized energy absorption and extension capacities of up to 57% and 26%, respectively). This study shows that designs generated with the 3DGAN model in lattice structures improve the design space by improving their mechanical properties. Moreover, the successful integration of deep learning and SLM-AM opens up new possibilities for creating custom parts with improved strength-to-weight ratios, dimensional accuracy, and mechanical performance. [ABSTRACT FROM AUTHOR]

In recent years, the European Commission and the International Maritime Organization (IMO) implemented various operational measures and policies to reduce ship fuel consumption and related emissions. The effectiveness of these measures relies upon developing accurate predictive models encompassing the influence of real operational conditions. This paper presents a deep learning method for the prediction of ship fuel consumption. The method utilizes big data analytics from sensors, voyage reporting and hydrometeorological data, comprising of 266 variables made available following sea trials of a Kamsarmax bulk carrier of Laskaridis Shipping Co. Ltd. A variable importance estimation model using a Decision Tree (DT) is used to understand the underlying relationships in the available dataset. Consequently, a deep learning model is developed to understand the influence of sailing speed, heading, displacement/draft, trim, weather, sea conditions, etc. on ship fuel consumption (SFC). This is achieved by incorporating attention mechanism into Bi-directional Long Short-Term Memory (Bi-LSTM) network. The potential of the new method is demonstrated by training data streams corresponding to real ship fuel consumption rates as well as internal and external operational conditions. A comprehensive comparison with existing methods indicates that the Bi-LSTM with attention mechanism presents the best fit when using high frequency data. It is concluded that subject to further testing and validation the method could be used for the development of decision support systems for monitoring environmentally sustainable ship operations. [ABSTRACT FROM AUTHOR]