Search

A tidal turbine can benefit from exquisitely designed morphing blades with a flexible trailing edge by mitigating up to 90% of the load fluctuation in harsh ocean environments, which reduces the overall cost of tidal energy. However, existing fluid–structure interaction (FSI) methods of resolving flow-induced deformation of the blades is computationally expensive, which poses an important challenge to effective morphing blade design. This paper presents a novel static FSI tool based on deep learning to cost-efficiently analyze the fluid–structure coupling of a hydrofoil. Specifically, adopting a convolutional neural network (CNN) to predict the fluid force and finite element method (FEM) to solve the solid structure response, a new CNN-FEM framework with an iterative scheme for solving the FSI problem is developed to achieve equilibrium between the fluid and structural forces. The new framework is used to predict the elastic deformation of the flexible blade section of the hydrofoil to demonstrate its effectiveness in the FSI evaluation. Comparison of the results to those produced by commercially developed software (i.e., Ansys Workbench) shows that this method yields extremely close prediction results of average equivalent stress and an accuracy of more than 92%. Moreover, it is 100 times more computationally efficient than the commercial Ansys Workbench software, requiring less than 3s for one-way FSI calculation. Taking advantage of this cost-effectiveness, the CNN-FEM can be used to achieve the accurate prediction of the deformation characteristics of the flexible hydrofoil under various flow scenarios that lay a foundation for advanced morphing blade design in the future. [ABSTRACT FROM AUTHOR]

The existing 3D deep learning methods adopt either individual point-based features or local-neighboring voxel-based features, and demonstrate great potential for processing 3D data. However, the point-based models are inefficient due to the unordered nature of point clouds and the voxel-based models suffer from large information loss. Motivated by the success of recent point-voxel representation, such as PVCNN and DRINet, we propose a new convolutional neural network, called Multi Point-Voxel Convolution (MPVConv), for deep learning on point clouds. Integrating both the advantages of voxel and point-based methods, MPVConv can effectively increase the neighboring collection between point-based features and also promote independence among voxel-based features. Extensive experiments on benchmark datasets such as ShapeNet Part, S3DIS and KITTI for various tasks show that MPVConv improves the accuracy of the backbone (PointNet) by up to 36% , and achieves higher accuracy than the voxel-based model with up to 34 × speedups. In addition, MPVConv outperforms the state-of-the-art point-based models with up to 8 × speedups. Also, our MPVConv only needs 65% of the GPU memory required by the latest point-voxel-based model (DRINet). The source code of our method is attached in https://github.com/NWUzhouwei/MPVConv. [Display omitted] • In this paper, we propose Multi Point-Voxel Convolution (MPVConv) which takes the advantages of both the voxel- and point-based methods, and can work with different backbones for a wide range of 3D tasks. • Our MPVConv increases not only the neighboring collection for point-based features, but also the independence among voxel-based features. • 1×1×1 3D CNN is adopted in fusion module to strengthen the information association and nonlinearity between point-based and voxel-based features. • We replace the tri-linear interpolation with nearest-neighbor interpolation to reduce the computation cost of voxel feature interpolation. [ABSTRACT FROM AUTHOR]

Forming a UAV cluster with multiple UAVs to collaborate in accomplishing tasks is the future development of battlefield. The UAV cluster can collaborate by communicating through inter-aircraft links and can quickly and accurately perform complex tasks such as path planning, collaborative reconnaissance, collaborative sensing and attack. In the process of Unmanned Aerial Vehicle (UAV) cooperative operation, obtaining the real-time attitude information of each UAV is preliminary to implement collaborative cooperation. However, in practice, due to the internal malfunction of sensors and mutual interference among UAVs, the acquired attitudes often suffer from data missing. This paper proposed a novel missing data reconstruction method for UAV's attitude in the case of sensor failures. The attitude data of UAV is obtained through an advanced UAV simulation platform AirSim. Through fusing the temporal convolutional network (TCN) and Bi-directional long short-term memory (Bi-LSTM), a deep regression framework is established for missing data construction. In addition, we evaluate the reliability of the proposed method by comparing with different baseline models and different combinations of data missing. The performance of our method is quantified using metrics of root-mean-square-error (RMSE), mean absolute error (MAE), determination coefficient (DC), mean absolute percentage error (MAPE), and symmetric mean absolute percentage error (SMAPE) score. The averaged performance values of our proposed method outperform other baseline algorithms. The results of the experiment show that the study could well-address the issue of missing data reconstruction and provide reliable attitude data. • A deep learning framework for UAV's missing data reconstruction is proposed. • TCN and Bi-LSTM is fused to reconstruct the missing data of UAV's attitude. • The performance for missing data reconstruction under different sensor failures is evaluated. [ABSTRACT FROM AUTHOR]

Reconstructing natural image from its corresponding compressive sensing (CS) measurements is an ill-posed problem. Learning accurate prior of desirable image is essential to solve this inverse problem in high quality, especially for CS reconstruction from noisy measurements. The existing learning-based methods cannot effectively simulate whole potential noise in the testing data during external learning, and cannot effectively exploit the information from internal testing data. In this paper, we present an effective convolutional neural network (CNN) based method for CS reconstruction from noisy measurements, which learns the deep prior from an external dataset and internal noisy testing data with Stein's unbiased risk estimator (SURE). Specifically, we first pre-train an arbitrary CNN for CS reconstruction with an external dataset to learn a common prior. Then, we utilize meta learning to find a generic initial parameter that is suitable for fast internal learning and adaption to various noisy measurements. Finally, we customize the learned network to learn a specific prior for each internal testing data under Gaussian noise or more general mixed Poisson-Gaussian noise. Experimental results show that the proposed method outperforms the state-of-the-art methods under both comprehensively quantitative metrics and perceptive quality. [ABSTRACT FROM AUTHOR]

Filter pruning has drawn extensive attention due to its advantage in reducing computational costs and memory requirements of deep convolutional neural networks. However, most existing methods only prune filters based on their intrinsic properties or spatial feature maps, ignoring the correlation between filters. In this paper, we suggest the correlation is valuable and consider it from a novel view: the frequency domain. Specifically, we first transfer features to the frequency domain by Discrete Cosine Transform (DCT). Then, for each feature map, we compute a uniqueness score, which measures its probability of being replaced by others. This way allows to prune the filters corresponding to the low-uniqueness maps without significant performance degradation. Compared to the methods focusing on intrinsic properties, our proposed method introduces a more comprehensive criterion to prune filters, further improving the network compactness while preserving good performance. In addition, our method is more robust against noise than the spatial ones since the critical clues for pruning are more concentrated after DCT. Experimental results demonstrate the superiority of our method. To be specific, our method outperforms the baseline ResNet-56 by 0.38 % on CIFAR-10 while reducing the floating-point operations (FLOPs) by 47.4 %. In addition, a consistent improvement can be observed when pruning the baseline ResNet-110: 0.23 % performance increase and up to 71 % FLOPs drop. Finally, on ImageNet, our method reduces the FLOPs of the baseline ResNet-50 by 48.7 % with only 0.32 % accuracy loss. [ABSTRACT FROM AUTHOR]

• We propose a new graph convolution network with subgraph embedding. • It extracts local information with subgraphs partitioned by Girvan Newman algorithm. • Experiments confirm the superiority of predicting mutagenicity in aromatic hydrocarbons. • The accuracy is verified with comparison of nine SOTA deep learning models. • The proposed method prevents about 15 %p of GCN's information dilution. An aromatic hydrocarbon refers to an organic material having a carbon ring such as benzene and a functional group in the carbon ring. As the industry develops, natural pollution becomes harsh, new compounds emerge, and the exposure to aromatic hydrocarbons is continuously increasing. Predicting mutagenicity is one of the crucial issues in reducing the risk because these organisms may have properties that penetrate the DNA of living things to cause mutations. Recently, the accuracy of mutation prediction has improved due to the power of deep learning. However, most conventional methods do not consider the characteristics of molecular aromatic hydrocarbons, which dilutes local information and results in a severe deterioration of the prediction performance. In this paper, we propose a method of exploiting subgraph convolution neural networks that enables the extraction of local information of a graph by partitioning it to maintain the detailed information. For extracting the features of molecules, we use the Girvan Newman algorithm to partition the graph according to the carbon ring and functional group and obtain the embedding vectors of the subgraphs as well as the original graph with graph convolution network (GCN). The embedding vectors are combined to represent the whole graph information and predict mutagenicity. Experiments with MUTAG, NCI1 and NCI109, datasets for predicting mutagenicity of molecules in graph structure, confirm that we successfully segment carbon rings and functional groups from molecular graphs and predict mutations using the partitioned graphs, leading to a 2 %p performance improvement. In addition, the proposed method has prevented about 15 %p of information dilution in GCN, and an analysis of the latent space of graphs reveals that the subgraphs extracted maintain the local information appropriately. [ABSTRACT FROM AUTHOR]

• Introduces the novel task of Sexual Facial Expression Recognition (SFER). • Triple-BigGAN; a conditional GAN with a generator, discriminator, and classifier. • SEA-Faces-30k; a new dataset with Erotic, Suggestive-Erotic, and Non-Erotic classes. • Evaluation using FER-2013, ExpW, MNIST, CIFAR-10, SVHN, and SEA-Faces-30k datasets. Automatic recognition of facial images showing erotic expressions can help to understand our social interaction and to detect non-appropriate images even when there is no nakedness present in them. This paper contemplates, for the first time, to exploit facial cues applied to automatic Sexual Facial Expression Recognition (SFER). With this goal, we introduce a new dataset named Sexual Expression and Activity Faces (SEA-Faces-30k) for SFER, which contains 30k manually labeled images under three categories: erotic, suggestive-erotic, and non-erotic. Deep Convolutional Neural Networks require large-scale annotated image datasets with diversity and variations to be properly trained. Unfortunately, gathering such a massive amount of data is not feasible in this area. Therefore, we present a new semi-supervised GAN framework named Triple-BigGAN, which learns a generative model and a classifier simultaneously. It learns both tasks in an end-to-end fashion while using unlabeled or partially labeled data. The Triple-BigGAN framework shows promising classification performance for the SFER task (i.e., 93.59 %) and other five benchmark datasets, i.e., FER-2013, CIFAR-10, Expression in-the-Wild (ExpW), Modified National Institute of Standards and Technology database (MNIST), and Street View House Numbers (SVHN). Next, we evaluated the quality of samples generated by Triple-BigGAN with a resolution of 256 × 256 pixels using Inception Score (IS) and Frechet Inception Distance (FID). Our approach obtained the best FID (i.e., 19.94 %) and IS (i.e., 97.98 %) scores on the SEA-Faces-30k dataset. Further, we empirically demonstrated that synthetic erotic face images generated by Triple-BigGAN could also help in improving the classification performance of deep supervised networks. [ABSTRACT FROM AUTHOR]

• A new approach for PD detection based on CNN deep features and i-vector features. • The impact of effective use of dynamic features for PD classification. • The significance of CNN deep features and i-vector features in terms of PD detection. Parkinson's disease (PD) corresponds to one of the most common neurological diseases in the world, which is mainly manifested by motor, cognitive, and language disorders. The change in the patient's voice is one of the most striking clinical signs and proves to be an element of interest to support the diagnosis and the assessment of PD. In this research paper, a new approach based on speech signal analysis is set forward to automatically detect Parkinson's disease. The approach evaluates two learning techniques, namely Support Vector Machines (SVM) and Convolutional Neural Networks (CNN), to classify data obtained from speech tasks. Two input data,i.e., the raw speech signal's values and the i-vector features of dimensions 100, 200 and 300 are extracted in this study. Eventually, an evaluation step is undertaken through the use of five evaluation metrics which are accuracy, precision, recall/sensitivity, specificity and f -score. The most pertinent obtained results for a test dataset composed of 28 participants are recorded as follows: an accuracy of 100%, precision of 0.99, recall/sensitivity of 0.98, specificity of 0.96 and f -score of 0.98. These findings corroborate that our approach can help in terms of PD detection. [ABSTRACT FROM AUTHOR]

• CNN architecture called MSFA-Net that acts as a texture feature extractor from raw images. • MSFA-Net requires to learn much fewer hyperparameters than state-of-the-art CNNs, and so reduces computation costs. • MSFA-Net outperforms several descriptors in multispectral texture classification and crop/weed recognition. • Texture features extracted from raw images are more discriminant than those extracted from demosaiced ones. Multispectral snapshot cameras fitted with a multispectral filter array (MSFA) acquire several spectral bands in one shot and provide a raw mosaic image in which a single channel value is available at each pixel. Texture features are classically extracted from fully-defined images that are estimated by demosaicing. Such an estimation may however cause spatio-spectral artifacts. Moreover, texture feature extraction becomes computationally inefficient and yields to high-dimensional features as the number of bands increases. In this paper, we propose an original approach based on a convolutional neural network called MSFA-Net to capture spatio-spectral interactions in raw images at reduced computation costs. Experiments of multispectral image classification and outdoor image segmentation show that the proposed approach outperforms several hand-crafted and deep learning-based feature extractors. [ABSTRACT FROM AUTHOR]

In recent years, deep learning (DL) methods have proven their efficiency for various computer vision (CV) tasks such as image classification, natural language processing, and object detection. However, training a DL model is expensive in terms of both complexities of the network structure and the amount of labeled data needed. In addition, the imbalance among available labeled data for different classes of interest may also adversely affect the model accuracy. This paper addresses these issues using a new convolutional neural network (CNN) based architecture. The proposed network incorporates both spatial and spectral information that combines two sub-networks: spatial-CNN and spectral-CNN. The spectral-CNN extracts spectral information, while spatial-CNN captures spatial information. Moreover, to make the features more robust, a multiscale spatial CNN architecture is introduced using different kernels. The final feature vector is formed by concatenating the outputs obtained from both spatial-CNN and spectral-CNN. To address the data imbalance problem, a generative adversarial network (GAN) was used to generate data for the underrepresented class. Finally, relatively a shallower network architecture was used to reduce the number of parameters in the network and improve the processing speed. The proposed model was trained and tested on Senitel-2 images for the classification of the debris-covered glacier. The results showed that the proposed method is well-suited for mapping and monitoring debris-covered glaciers at a large scale with high classification accuracy. In addition, we compared the proposed method with conventional machine learning approaches, support vector machine (SVM), random forest (RF) and multilayer perceptron (MLP). [ABSTRACT FROM AUTHOR]

When imaging at a long distance by ordinary visible cameras, the wavelength of visible light is easily interfered by fog or atmospheric effects, resulting in blurry or lost details in RGB images. However, NIR cameras are robust to the long distance imaging without color. In this paper, we propose long-range imaging using multispectral fusion of RGB and NIR images, called LRINet. We adopt unsupervised learning for the fusion to solve the absence of ground truth. LRINet is an end-to-end network based on convolutional neural networks (CNNs) that consists of three steps for the multispectral fusion: feature extraction, fusion and reconstruction. To align unpaired data and treat the discrepancy between RGB and NIR images, we construct WarpingNet to warp NIR features, and add it into the feature extraction. Since dark channel prior (DCP) provides distance from the camera by light transmission degree, we combine it with structure loss as the weight for fusion. To ensure the color fidelity, LRINet operates the fusion on the input of the RGB luma channel and NIR image. Experimental results show that LRINet produces natural-looking color images with clear details by successfully treating discrepancy between RGB and NIR images and outperforms state-of-art fusion models in term of both visual quality and quantitative measurements. • We propose long range imaging using multispectral fusion of RGB and NIR images, called LRINet. • LRINet consists of three steps for image fusion: feature extraction, fusion, and reconstruction. • We introduce WarpingNet into the feature extraction step to align the unpaired RGB and NIR images. • We combine DCP with structure loss to achieve long range imaging while keeping the original tone. • LRINet successfully treats discrepancy between two images and generates natural looking fusion images. [ABSTRACT FROM AUTHOR]

In this paper, we present an extensive review on deep learning methods for image restoration tasks. Deep learning techniques, led by convolutional neural networks, have received a great deal of attention in almost all areas of image processing, especially in image classification. However, image restoration is a fundamental and challenging topic and plays significant roles in image processing, understanding and representation. It typically addresses image deblurring, denoising, dehazing and super-resolution. There are substantial differences in the approaches and mechanisms in deep learning methods for image restoration. Discriminative learning based methods are able to deal with issues of learning a restoration mapping function effectively, while optimisation models based methods can further enhance the performance with certain learning constraints. In this paper, we offer a comparative study of deep learning techniques in image denoising, deblurring, dehazing, and super-resolution, and summarise the principles involved in these tasks from various supervised deep network architectures, residual or skip connection and receptive field to unsupervised autoencoder mechanisms. Image quality criteria are also reviewed and their roles in image restoration are assessed. Based on our analysis, we further present an efficient network for deblurring and a couple of multi-objective training functions for super-resolution restoration tasks. The proposed methods are compared extensively with the state-of-the-art methods with both quantitative and qualitative analyses. Finally, we point out potential challenges and directions for future research. [ABSTRACT FROM AUTHOR]

Infrared and visible image fusion aims to integrate complementary information from different types of images into one image. The existing image fusion methods are primarily based on convolutional neural network (CNN), which ignores long-range dependencies of images, resulting in the fusion network unable to generate images with good complementarity. Inspired by the importance of global information, we introduced the transformer technique into the CNN-based fusion network as a way to improve the entire image-level perception in complex fusion scenarios. In this paper, we propose an end-to-end image fusion framework based on transformer and hybrid feature extractor, which enables the network to focus on both global and local information, using the characteristics of transformer to compensate for the shortcomings of CNN itself. In our network, the dual-branch CNN module is used to extract the shallow features of images, and then the vision transformer module is used to obtain the global channel and spatial relationship in the features. Finally, the fusion results are obtained through the image reconstruction module. We calculate the loss in the features of different depths according to the different kinds of original images by using the pre-trained VGG19 network. The experimental results show the effectiveness of adding the vision transformer module. Compared with other traditional and deep learning methods, our method achieves state-of-the-art qualitative and quantitative experiments performance. [ABSTRACT FROM AUTHOR]

Analysis of point clouds through deep convolutional neural networks is an active area of research due to their massive real-world applications including autonomous driving, indoor navigation, robotics, virtual/augmented reality, unmanned aerial vehicles, and drone technology. However, capturing the fine-grained geometric and semantic properties for the underlying recognition task with raw unstructured point cloud is highly challenging due to the lack of explicit neighborhood relationship and sparsity among the points. In this paper, we have introduced a deep, hierarchical 3D point based architecture for object classification and part segmentation that is able to learn robust geometric features which remain invariant to both the geometry and orientation of the local patches. The proposed architecture consists of multiple layers of sampling, concentric annular convolution, pooling, and residual feature propagation blocks. In the skip connections of our deep residual design, we propose to use a combination of linear projection shortcut and nonlinear ReLU group normalization shortcut with batch normalization, to improve both the optimization landscape and the representational power. Our network achieves on par or even better than state-of-the-art results on synthetic and real-world object classification (i.e., ModelNet40 and ScanObjectNN) and part segmentation (i.e., ShapeNet-part) benchmark datasets. The implementation and results have been made available at ...https://github.com/Rabbia-Hassan/Deep_Annular_Residual_Feature_Learning_for_3dPointCloudsGitHub-link [ABSTRACT FROM AUTHOR]

The paper proposes a new class of nonlinear operators and a dual learning paradigm where optimization jointly concerns both linear convolutional weights and the parameters of these nonlinear operators. The nonlinear class proposed to perform a rich functional representation is composed by functions called rectified parametric sigmoid units. This class is constructed to benefit from the advantages of both sigmoid and rectified linear unit functions, while rejecting their respective drawbacks. Moreover, the analytic form of this new neural class involves scale, shift and shape parameters to obtain a wide range of activation shapes, including the standard rectified linear unit as a limit case. Parameters of this neural transfer class are considered as learnable for the sake of discovering the complex shapes that can contribute to solving machine learning issues. Performance achieved by the joint learning of convolutional and rectified parametric sigmoid learnable parameters are shown to be outstanding in both shallow and deep learning frameworks. This class opens new prospects with respect to machine learning in the sense that main learnable parameters are attached not only to linear transformations, but also to a wide range of nonlinear operators. [ABSTRACT FROM AUTHOR]

Segmentation of retinal vessel images is critical to the diagnosis of retinopathy. Recently, convolutional neural networks have shown significant ability to extract the blood vessel structure. However, it remains challenging to refined segmentation for the capillaries and the edges of retinal vessels due to thickness inconsistencies and blurry boundaries. In this paper, we propose a novel deep neural network for retinal vessel segmentation based on shared decoder and pyramid-like loss (SPNet) to address the above problems. Specifically, we introduce a decoder-sharing mechanism to capture multi-scale semantic information, where feature maps at diverse scales are decoded through a sequence of weight-sharing decoder modules. Also, to strengthen characterization on the capillaries and the edges of blood vessels, we define a residual pyramid architecture which decomposes the spatial information in the decoding phase. A pyramid-like loss function is designed to compensate possible segmentation errors progressively. Experimental results on public benchmarks show that the proposed method outperforms the backbone network and most state-of-the-art methods, especially in the regions of the capillaries and the vessel contours. In addition, performances on cross-datasets verify that SPNet shows stronger generalization ability. [ABSTRACT FROM AUTHOR]

The state-of-the-art performance for several real-world problems is currently reached by deep and, in particular, convolutional neural networks (CNN). Such learning models exploit recent results in the field of deep learning, leading to highly performing, yet very large neural networks with typically millions to billions of parameters. As a result, such models are often redundant and excessively oversized, with a detrimental effect on the environment in terms of unnecessary energy consumption and a limitation to their deployment on low-resource devices. The necessity for compression techniques able to reduce the number of model parameters and their resource demand is thereby increasingly felt by the research community. In this paper we propose the first extensive comparison, to the best of our knowledge, of the main lossy and structure-preserving approaches to compress pre-trained CNNs, applicable in principle to any existing model. Our study is intended to provide a first and preliminary guidance to choose the most suitable compression technique when there is the need to reduce the occupancy of pre-trained models. Both convolutional and fully-connected layers are included in the analysis. Our experiments involved two pre-trained state-of-the-art CNNs (proposed to solve classification or regression problems) and five benchmarks, and gave rise to important insights about the applicability and performance of such techniques w.r.t. the type of layer to be compressed and the category of problem tackled. [ABSTRACT FROM AUTHOR]

• We propose ReNAP for few-shot classification. • The APL module of ReNAP can learn class prototypes adaptively. • Experimental results demonstrate the superior classification performance of ReNAP. Traditional deep learning-based image classification methods often fail to recognize a new class that does not exist in the training dataset, particularly when the new class only has a small number of samples. Such a challenging and new learning problem is referred to as few-shot learning. In few-shot learning, the relation network (RelationNet) is a powerful method. However, in RelationNet and its state-of-the-art variants, the prototype of each class is obtained by a simple summation or average over the labeled samples. These simple sample statistics cannot accurately capture the distinct characteristics of the diverse classes of real-world images. To address this problem, in this paper, we propose the Relation Network with Adaptive Prototypical Learning method (ReNAP), which can learn the class prototypes adaptively and provide more accurate representations of the classes. More specifically, ReNAP embeds an adaptive prototypical learning module constructed by a convolutional network into RelationNet. Our ReNAP achieves superior classification performances to RelationNet and other state-of-the-art methods on four widely used benchmark datasets, FC100, CUB-200-2011, Stanford-Cars, and Stanford-Dogs. [ABSTRACT FROM AUTHOR]

Condition monitoring, fault diagnosis, and scheduled maintenance of wind turbines (WTs) are becoming a necessity to maximize their economic benefits and reduce their downtime. One of the critical faults in WTs is inter-turn short-circuit faults (ITSCF) in their rotating machines. Being dependent on higher wind speeds, offshore wind farms are more efficient than onshore ones, yet at the cost of higher ITSCF, more maintenance requirements, and difficulty in accessibility. Thus, they require an efficient non-contact fault diagnosis technique to enable operators to inspect all WT components with minimal contact. Among existing fault detection technologies, Infrared thermography (IRT) is considered as a non-invasive and non-destructive anomaly detection technique based on capturing thermal images by IR cameras, making it suitable for offshore harsh environments. Meanwhile, deep learning (DL) fault diagnosis approaches have proven to be promising intelligent tools with minimum human labor. This paper proposes an efficient and robust ensemble DL-based diagnosis method for ITSCF detection in induction rotating machines, besides identifying fault locations and short-circuit severity. The proposed technique is quite convenient for early diagnosis of offshore WT generator faults since it depends on IRT technology. Compared to previous studies, the proposed approach outperforms its counterparts since it integrates eight DL models, rather than using a single model, thus merging all their structural benefits altogether. Moreover, a cascaded feature fusion and selection procedure is introduced. First, the deep features extracted from the DL models are fused, then the most influential features are selected using a hybrid feature selection scheme. Thus, the classification accuracy is enhanced to reach 100% and meanwhile, the input features' size to the classifier is reduced, decreasing classification complexity and training time. Finally, no clustering or segmentation phases are required in the proposed technique, resulting in a further decrease in the diagnosis time and computation burden. • A CAD is proposed for faults diagnosis in induction generator-based offshore windfarms. • It fuses 8 DL models instead of using an individual one, thus boosting the classification ability. • The CAD employs a few features to perform diagnosis through a cascaded procedure. • First, features are fused, then significant features are selected via hybrid feature selection. • No segmentation or clustering stages are required which adds to the system's simplicity. [ABSTRACT FROM AUTHOR]

Low-light image enhancement studies how to improve the quality of images captured under poor lighting conditions, which is of real-world importance. Currently, convolutional neural network (CNN)-based methods with state-of-the-art performance have become the mainstream of research. However, most CNN-based methods improve the performance of the algorithm by increasing the width and depth of the neural network, which requires large computing device resources. In this paper, we propose a knowledge distillation method for low light image enhancement. The proposed method uses a teacher-student framework in which the teacher network tries to transfer the rich knowledge to the student network. The student network learns the knowledge of image enhancement under the supervision of ground truth images and under the guidance of the teacher network simultaneously. Knowledge transfer between the teacher-student network is accomplished by distillation loss based on attention maps. We designed a gradient-guided low-light image enhancement network that can be divided into an enhancement branch and a gradient branch, where the enhancement branch is learned under the guidance of the gradient branch to better preserve structural information. The teacher and student networks use a similar structure, but they have different model sizes. The teacher network has more parameters and more powerful learning capabilities than the student network. With the help of knowledge distillation, our approach can improve the performance of the student network without increasing the computational burden during the testing phase. The qualitative and quantitative experimental results demonstrate the superiority of our method compared to the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• Basic bibliometric indexes are utilized to characterize the global trends of medical image research from 1993 to 2022. • Network-based methods are employed to analyze the co-word, co-authorship, and co-citation relations. • Analysis shows the present medical image research is on an upward trend. • Medical image research starts to speed up after 2015 due to the application of convolutional neural networks. With the development of artificial intelligence and high-performance computing equipment, new technologies have huge effects on medical image research. However, it is difficult to find out when new research topics appear, who those authors with influences are, and how relevant publications influence the academic community. In order to catch up with global research trends, traditional methods of literature review are inadequate to acquire information. In this case, a data-driven analysis offers a new quantitative approach to studying global research trends. Specifically, this paper used several basic bibliometric indexes to characterize the global trends of medical image research from 1993 to 2022, including yearly output, active journals, important authors, active institutions, and main countries. Furthermore, we utilized network-based methods to analyze the internal relations of co-word, co-authorship, and co-citation, so as to discover academic hotspots and clarify global research trends. Finally, some conclusions are drawn as follow: (1) The present medical image research is on an upward trend. The number of publications on medical image surged since 2015 due to advances in deep learning. Deep learning and convolutional neural networks (CNNs) are both popular research keywords in recent years. (2) IEEE Transactions on Medical Imaging is the most influential journal in view of Total Local Citation Score (TLCS) and Total Global Citation Score (TGCS), followed by Medical Image Analysis. Neurocomputing and Information Fusion are well-recognized in local research community. (3) Van Ginneken B and Aerts HJWL are representative scholars in consideration of TLCS and TGCS. (4) The USA is a leading country in medical image research. Other influential countries include China, India, UK, Germany, France, Canada, Netherlands, Australia, Italy, South Korea, Switzerland, etc. Most important institutions are from these countries, including Harvard , UMich , Stanford , UPenn , UNC , CAS , SJTU , UCL , UofT , RU , etc. (5) Application of artificial intelligence technologies, especially CNNs, has dramatically promoted global studies of medical image since 2015. Interdisciplinary collaborations become popular among experts with different disciplines backgrounds. We can infer that medical image analysis and application based on deep learning will still be a flourishing field in the near future with the improvement of algorithms and the application of high-performance computing equipment. [ABSTRACT FROM AUTHOR]

Fine-grained visual categorization is an important task in computer vision. Prior works on fine-grained visual categorization have paid much attention to addressing intra-class variation and inter-class similarity. However, they rarely study that task from the perspective of probability distribution. In this paper, we propose a novel refined probability distribution module based on deep convolutional neural network. Our module computes the probability of an image by fully utilizing the similarity information between images. Firstly, we use deep neural networks to obtain the initial probability distribution and extract features. Then, we build a network whose inputs are features for calculating image-to-image similarity scores. Finally, our module refines the initial probability distribution based on an effective batch random walk operation with similarity scores. Our module can be plugged into many deep convolutional neural networks. Experimental results show that our approach outperforms state-of-the-art methods on the CUB-200–2011, FGVC-Aircraft and Stanford Cars datasets respectively. [ABSTRACT FROM AUTHOR]

• A novel deep learning model for laser-induced damage change detection is proposed. • The proposed model combines siamese CNN with visual attention mechanism. • A novel SoftmaxFocal loss is put forward to train the network. • Our model outperforms four different methods overall on three real datasets. With high-energy laser irradiating, the laser-induced damages may occur in the surfaces of optical elements in laser facilities. As the laser-induced damage changes can badly affect regular and healthy operation of laser facilities, it is essential to effectively detect real damage changes while suppressing meaningless and spurious changes in captured optical images. In order to achieve high-precision laser-induced damage change detection, this paper presents a novel deep learning model which exploits visual attention-based siamese convolutional neural network with SoftmaxFocal loss and significantly improves the performance of damage change detection. In the proposed model, an end-to-end classification network is designed and trained which fuses the spatial-channel domain collaborative attention modules into siamese convolutional neural network thus achieving more efficient feature extraction and representation. For the purpose of addressing the unbalanced distribution of hard and easy samples, a novel loss function which is termed as SoftmaxFocal loss is put forward to train the proposed network. The SoftmaxFocal loss creatively introduces an additive focusing term into original softmax loss which greatly enhances the online hard sample mining ability of the proposed model. Experiments conducted on three real datasets demonstrate the validity and superiority of the proposed model. [ABSTRACT FROM AUTHOR]

The combination of convolutional and recurrent neural networks is a promising framework. This arrangement allows the extraction of high-quality spatio-temporal features together with their temporal dependencies. This fact is key for time series prediction problems such as forecasting, classification or anomaly detection, amongst others. In this paper, the TSFE DL library is introduced. It compiles 22 state-of-the-art methods for both time series feature extraction and prediction, employing convolutional and recurrent deep neural networks for its use in several data mining tasks. The library is built upon a set of Tensorflow+Keras and PyTorch modules under the AGPLv3 license. The performance validation of the architectures included in this proposal confirms the usefulness of this Python package. [ABSTRACT FROM AUTHOR]

Acquiring incomplete k-space matrices is an effective way to accelerate Magnetic Resonance Imaging (MRI). It is an important and challenging task to accurately reconstruct images from such under-sampled k-space matrices. On the one hand, neither image-domain oriented nor frequency-domain oriented deep Convolutional Neural Networks can simultaneously employ both frequency features and spatial features for cooperatively improving reconstruction accuracy. On the other hand, existing dual-domain reconstruction methods adopt heavy encoder-decoder frameworks, resulting in low efficiency and information loss in the process of pooling. To deal with these problems, in this paper, we propose a full-resolution dual-domain reconstruction network, called DIIK-Net. The DIIK-Net consists of a full-resolution frequency-domain branch, a full-resolution image-domain branch, and cross-domain interaction modules between the two branches. The first novelty of the proposed method is that the features of each block of frequency-domain branch are extracted by 1 × 1 filters, which reduces computational cost and captures rich contextual information. Due to the fact that an element in frequency domain conveys information of the whole image, 1 × 1 convolutional blocks are able to extract large contextual information with the interaction of image domain. The second novelty is that the image-domain branch consists of a very small number of 3 × 3 convolutional blocks and each block has very large field of perception due to integration of frequency domain. The third novelty lies in the simple and effective cross-domain interaction module. Experimental results on the challenging fastMRI dataset demonstrate that the proposed method is capable of achieving higher reconstruction accuracy with a few number of parameters. [ABSTRACT FROM AUTHOR]

Accurate and automatic segmentation of pancreatic tumors and organs from medical images is important for clinical diagnoses and making treatment plans for patients with pancreatic cancer. Although deep learning methods have been widely adopted for this task, the segmentation accuracy, especially for pancreatic tumors, still needs to be further improved because (1) phenotypic differences, such as volumes, tend to make the models focus on pancreatic learning, resulting in insufficient tumor feature selection; (2) deep learning models may fall into local optima, leading to unsatisfactory segmentation results for tumors and pancreas. To alleviate the above issues, in this paper, we propose a 3D fully convolutional neural network with three temperature guided modules, namely, balance temperature loss, rigid temperature optimizer and soft temperature indictor, to realize joint segmentation of the pancreas and tumors. Specifically, balance temperature loss is designed to dynamically adjust the learning points between tumors and the pancreas to balance the selected features, and it is aimed at improving the accuracy of tumor segmentation without losing pancreas information. Rigid temperature optimizer is proposed to accept nonimproving moves probabilistically to adaptively avoid local optima. To further refine the segmentation results, we propose the soft temperature indictor to guide the network into a fine-tuning state automatically when the model tends to stability. Our experimental results are more accurate than the fourteen top-ranking methods in pancreas and tumors segmentation on the MSD pancreas dataset and six top-ranking methods in brain tumors segmentation. Ablation studies verify the effectiveness of the three temperature guided modules. [ABSTRACT FROM AUTHOR]

Residual Network (ResNet) achieves deeper and wider networks with high-performance gains, representing a powerful convolutional neural network architecture. In this paper, we propose architectural refinements to ResNet that address the information flow through several layers of the network, including the input stem, downsampling block, projection shortcut, and identity blocks. We will show that our collective refinements facilitate stable backpropagation by preserving the norm of the error gradient within the residual blocks, which can reduce the optimization difficulties of training very deep networks. Our proposed modifications enhance the learning dynamics, resulting in high accuracy and inference performance by enforcing norm-preservation throughout the network training. The effectiveness of our method is verified by extensive experimental results on five computer vision tasks, including image classification (ImageNet and CIFAR-100), video classification (Kinetics-400), multi-label image recognition (MS-COCO), object detection and semantic segmentation (PASCAL VOC). We also empirically show consistent improvements in generalization performance when applying our modifications over different networks to provide new insights and inspire new architectures. The source code is publicly available at: https://github.com/bharatmahaur/LeNo. [ABSTRACT FROM AUTHOR]

Inspired by the successful use of deep learning in computer vision, in this paper we introduce ForCNN , a novel deep learning method for univariate time series forecasting that mixes convolutional and dense layers in a single neural network. Instead of using conventional, numeric representations of time series data as input to the network, the proposed method considers visual representations of it in the form of images to directly produce point forecasts. Three variants of deep convolutional neural networks are examined to process the images, the first based on VGG-19, the second on ResNet-50, while the third on a self-designed architecture. The performance of the proposed approach is evaluated using time series of the M3 and M4 forecasting competitions. Our results suggest that image-based time series forecasting methods can outperform both standard and state-of-the-art forecasting models. • An image-based deep learning method for time series forecasting is proposed. • Numeric time series data are converted to images using simple line plots. • Deep convolutional neural networks are employed for producing forecasts from each image. • The proposed, image-based, method outperforms state-of-the-art forecasting methods. [ABSTRACT FROM AUTHOR]

Gene Expression Data is the biological data to extract meaningful hidden information from the gene dataset. This gene information is used for disease diagnosis especially in cancer treatment based on the variations in gene expression levels. DNA microarray is an efficient method for gene expression classification and prediction of cancer disease for specific types of cancer. Due to the abundance of computing power, deep learning (DL) has become a widespread technique in the healthcare sector. The gene expression dataset has a limited number of samples but a large number of features. Data augmentation is needed for gene expression datasets to overcome the dimensionality problem in gene data. It is a technique to generating the synthetic samples to increase the diversity of data. Deep learning methods are designed to learn and extract the features that come from the raw input data in the form of multidimensional arrays. This paper reviews the existing research in deep learning techniques like Feed Forward Neural Network (FFN), Convolutional Neural Network (CNN), Autoencoder (AE) and Recurrent Neural Network (RNN) for the classification and prediction of cancer disease and its types through gene expression data analysis. [ABSTRACT FROM AUTHOR]

An accurate, frequently updated, automatic and reproducible mapping procedure to identify seasonal cultivated crops is a prerequisite for many crop monitoring activities. Deep learning was demonstrated to be an effective mapping approach already successfully applied to decametric resolution satellite images (like Sentinel-2 data) to produce yearly crop maps. In this framework, algorithm training is performed with ground truth typically consisting of spatially explicit information available after the end of the season (e.g. yearly crop maps and/or farmer declaration for subsidies at parcel level); however, such data (i) does not allow performing in-season prediction, and (ii) does not provide temporal details fundamental to describe a dynamic crop succession and/or to understand crop management (i.e. planting and harvesting). In this paper we present a Deep Neural Network-based approach capable of generating (i) a crop map of the current season at a specific point in time ("In season mapping" conventionally at the end of the current year), along with (ii) all intermediate maps during the season able to describe in near real-time the evolution of crop presence ("Dynamic-mapping" at the temporal granularity of satellite imagery revisiting, e.g., 5 days for Sentinel-2 data). This approach adopts a smart training procedure of a Deep Neural model by exploiting historical satellite data and ground truth. We introduce a method to automatically generate "short-term" ground truth maps (i.e. 5 days reference) starting from the "long-term" ones (i.e. available yearly static reference) and characterizing temporally the different crop presence by performing a phenological analysis of historical time series. The model was trained and validated in Lombardy (North of Italy) exploiting multi-annual authoritative crop maps from 2016 to 2019. Validation was performed both in time (same areas used for training in a different year) and space (different location) for the year 2019. The quantitative error metrics calculation and Spatio-temporal analysis clearly demonstrate that the model can predict in-season crop presence with a generalization capacity over the long-term (yearly maps: OA > 70% and Kappa > 0.64%) and that the short-term predictions (5 days maps) are coherent with the reference information from expert knowledge (local crop calendars). The model can produce dynamically along the season short-term maps with a medium-high crop-specific User Accuracy at the maximum green-up phase (UA > 53% up to 95%). These products are of extreme interest for final users providing information at the peak of plant development that dynamically changes according to the considered crop, the specific location and the investigated season. These results demonstrate that it is possible to produce a crop map early in the season and extract useful additional information such as crop intensity (e.g. double crops presence) and crop dynamics related to different sowing dates. [ABSTRACT FROM AUTHOR]

• Thispaperfirstlyproposed a supervised-unsupervised combinednetwork for high-fidelitypansharpening. • Thispaperproposed a supervised networkby multiscale mechanisms, dilated convolution and skip connection. • This paperproposeda universalunsupervisedspatial-spectral joint compensationnetwork. Deep learning for pansharpening method has become a hot research topic in recent years due to the impressive performance, and the convolutional neural networks (CNN)-based pansharpening methods on Wald's protocol (i.e., the general adoption of the network learned at a coarser reduced resolution scale to the finer full resolution) have been dominating for a long time in this research area. However, the scale-invariant assumption may not be accurate enough to make full use of the spatial and spectral information of original panchromatic (PAN) and multispectral (MS) images at full resolution. In this paper, a Supervised-Unsupervised combined Fusion Network (SUFNet) for high-fidelity pansharpening is proposed to alleviate this problem. First, by comprehensively considering the robustness of the network with reference label images, a novel supervised network based on Wald's protocol is proposed by integrating the multiscale mechanisms, dilated convolution, and skip connection, termed SMDSNet. Then, an interesting Unsupervised Spatial-Spectral Compensation Network (USSCNet) without real high-spatial-resolution (HR) MS label image is proposed to enhance the spatial and spectral fidelity of the SMDSNet. The qualitative and quantitative results in reduced resolution and full resolution experiments on different satellite datasets demonstrate the competitive performance of the proposed method. Furthermore, the proposed USSCNet can be employed as a universal spatial-spectral compensation framework for other pansharpening methods. [ABSTRACT FROM AUTHOR]

Distracted driving is one of the key factors that cause drivers to ignore potential road hazards and then lead to accidents. Existing efforts in distracted behavior recognition are mainly based on deep learning (DL) methods, which identifies distracted behaviors by analyzing static characteristics of images. However, the convolutional neural network (CNN) — based DL methods lack the causal reasoning ability for behavior patterns. The uncertainty of driving behaviors, noise of the collected data, and occlusion between body agents, bring additional challenges to existing DL methods to recognize distracted behaviors continuously and accurately. Therefore, in this paper, we propose a distracted behavior recognition method based on the Temporal–Spatial double-line DL network (TSD-DLN) and causal And-or graph (C-AOG). TSD-DLN fuses the attention feature extracted from the dynamic optical flow information and the spatial feature of the single video frame to recognize the distracted driving posture. Furthermore, a causal knowledge fence based on C-AOG is fused with TSD-DLN to improve the recognition robustness. The C-AOG represents the causality of behavior state fluent change and adopts counterfactual reasoning to suppress behavior recognition failures caused by frame features distortion or occlusion between body agents. We compared the performance of the proposed method with other state-of-the-art (SOTA) DL methods on two public datasets and self-collected dataset. Experimental results demonstrate that proposed method significantly outperforms other SOTA methods when acquiring distracted driving behavior by processing consecutive frames. In addition, the proposed method exhibits accurate continuous recognition and robustness under incomplete observation scenarios. • A novel behavior recognition model based on multi-features fusion is proposed. • A hierarchical causality graph is designed for behavior fluent inferring. • Counterfactual reasoning mechanism for misrecognition suppression is proposed. • A new behavior dataset representing the distracted driving process is constructed. [ABSTRACT FROM AUTHOR]

The detection of underwater fish targets is critical for ecological monitoring and marine biodiversity research. However, underwater fish detection is typically constrained by problems including low image quality and variable underwater surroundings. On behalf of further improving the underwater fish detection accuracy in complex underwater environments, this paper proposes a dual-path (DP) Pyramid Vision Transformer (PVT) feature extraction network named DP-FishNet. The backbone network DP-PVT composed from the PVT Network is made up of two feature extraction paths. The first represents the Vision Transformer path, which extracts global features to enhance the distinction between the foreground and background of underwater images. The second is the convolutional neural network path, which enhances the accuracy of detecting small targets by extracting local features. Additionally, to more effectively utilize the feature information extracted by the network, this paper provides a promising solution to employ the content-aware reassembly of features (Carafe) in the feature pyramid network (FPN). The seesaw loss is utilized as a classification loss to address the problem of unbalanced samples caused by the gap in the number of fish populations. According to the experimental findings, the AP and AP50 of the DP-FishNet are 76.0% and 95.2%, respectively. In comparison to currently available advanced two-stage detection algorithms, the quantity of computation and parameters is reduced by approximately 40%. DP-FishNet strengthens the ability to extract global and local features from underwater images and enhances feature reuse. DP-FishNet can be utilized to detect fish targets in actual and complicated underwater habitats. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new deformable convolutional network with attention mechanism to deal with the Chinese character of various dynasties. These ancient Chinese characters are hieroglyph with special spatial structure and gradually evolving into modern Chinese characters. However, because of the intrinsic limitations of traditional convolutional networks in the model, the geometric structure of the convolutional module is fixed. Therefore, we build a new architecture called attention-based deformable networks for this task. Besides, we construct a new dataset called the Chinese characters from Various Dynasties Dataset (CCDD), which includes the evolution of Chinese characters between major dynasties. Deformable convolution based on attention mechanism can obtain those more important offsets, resulting in better performance than the original deformable convolution. In the experimental stage, extensive experiments validate the performance of our approach. Firstly, the effectiveness of this network was verified on two text datasets, Mnist(99.38%) and notMnist(98.22%) both comparing with several modules. Finally, the best performance was achieved on the CCDD dataset(91.50%) compared to other classical deep learning networks. This is the first paper to apply deep learning methods to recognize and classify the characters of major dynasties in China. And archaeological workers need to be able to recognize Chinese characters from various periods in China, and our algorithm can assist them in completing this task to a certain extent. It can also help cultural relic enthusiasts roughly determine the age of cultural relics. [ABSTRACT FROM AUTHOR]

Marine ecosystems and their fish habitats are becoming increasingly important due to their integral role in providing a valuable food source and conservation outcomes. Due to their remote and difficult to access nature, marine environments and fish habitats are often monitored using underwater cameras to record videos and images for understanding fish life and ecology, as well as for preserve the environment. There are currently many permanent underwater camera systems deployed at different places around the globe. In addition, there exists numerous studies that use temporary cameras to survey fish habitats. These cameras generate a massive volume of digital data, which cannot be efficiently analysed by current manual processing methods, which involve a human observer. Deep Learning (DL) is a cutting-edge Artificial Intelligence (AI) technology that has demonstrated unprecedented performance in analysing visual data. Despite its application to a myriad of domains, its use in underwater fish habitat monitoring remains under explored. In this paper, we provide a tutorial that covers the key concepts of DL, which help the reader grasp a high-level understanding of how DL works. The tutorial also explains a step-by-step procedure on how DL algorithms should be developed for challenging applications such as underwater fish monitoring. In addition, we provide a comprehensive survey of key deep learning techniques for fish habitat monitoring including classification, counting, localisation, and segmentation. Furthermore, we survey publicly available underwater fish datasets, and compare various DL techniques in the underwater fish monitoring domains. We also discuss some challenges and opportunities in the emerging field of deep learning for fish habitat processing. This paper is written to serve as a tutorial for marine scientists who would like to grasp a high-level understanding of DL, develop it for their applications by following our step-by-step tutorial, and see how it is evolving to facilitate their research efforts. At the same time, it is suitable for computer scientists who would like to survey state-of-the-art DL-based methodologies for fish habitat monitoring. • Marine environments and fish habitats are widely surveyed using underwater cameras. • Deep learning (DL) offers an effective solution for processing underwater videos. • We write a tutorial on the key concepts and steps of DL for underwater monitoring. • We provide a survey of key DL-based visual fish monitoring methods/datasets. • We shed light on challenges/opportunities of DL for underwater fish monitoring. [ABSTRACT FROM AUTHOR]

This paper considers the multi-label thoracic abnormality classification with chest X-ray images. In clinical settings, Chest X-ray imaging is a general diagnostic tool applied to visualize numerous thoracic pathological changes. While deep learning techniques have been extensively tested in this field, certain challenges persist. The data in existing thoracic abnormality datasets is insufficient, and some diseases are extremely imbalanced. Meanwhile, the dependencies between different labels are often ignored. To tackle these issues head-on, this paper introduces two crucial modules: the group-wise spatial attention (GWSA) module and the label co-occurrence dependency (LCD) module, integrated with DenseNet121 backbone. Specifically, GWSA enhances the spatial features within distinct groups while keeping the between-group feature discrimination. LCD models the correlations between different thoracic abnormalities to refine the predicted probabilities. In conjunction with the DenseNet121 backbone, these two modules reach an average AUC score of 0.818 on Chest X-ray14 dataset, achieving state-of-the-art. Source code is available at https://github.com/YujiaKCL/Chest-Xray14-GWSA-LCD. [ABSTRACT FROM AUTHOR]

In this paper, machine learning (ML)/deep learning (DL) has been employed to predict the thermal runaway (TR) occurrence in Li-ion batteries (LIBs). State-of-the-art convolutional neural networks (CNNs) are adopted to forecast the evolution of the TR phenomenon with a classification approach of three distinct stages, namely, safe operation, critical condition of triggered the TR, and the actual TR occurrence. In addition, the "you look only once" (YOLO) object detection ML model is used to identify the location of the TR heat sources within the battery. The neural networks are trained on simulated thermal images, computed with the multiphysics modeling of LIBs. The multiphysics modeling approach comprises a coupled thermal, electrochemical (P2D model), and degradation sub-models. The degradation phenomenon leading to the TR considered in this paper is the solid electrolyte interface (SEI) formation/decomposition on the negative electrode. The proposed ML model exhibits high accuracy in predicting the TR evolution stages and heat source locations. The combined multiphysics and ML modeling approach developed in this work provides qualitative insights for 'on-the-fly' prediction of the TR, as well as a framework for extending data-driven methodologies to broad applications in electrochemistry. • Multiphysics modeling of thermal runaway (TR). • Simulated battery thermal imaging for training machine learning models for TR detection. • Multiclass classification of TR evolution: safe, critical, and TR stages. • Real-time detection of TR heat sources using object detection. [ABSTRACT FROM AUTHOR]

• This paper proposed a novel deep graph learning framework for airspace complexity prediction that can comprehensively capture the spatio-temporal dependencies between large-scale airspace sectors. • This paper constructed a multimodal adaptive graph convolution module that can effectively explore the multiple spatial dependencies and adaptively adjust the impact of different spatial modes on airspace complexity in a data-driven learning manner. • This paper designed a temporal convolution module based on a multiple-time-step self-attention mechanism to extract the mixed short- and long-term temporal patterns from both local and global perspectives. • This paper constructed a comprehensive dataset from real-world air traffic data within Chinese domestic airspace, ensuring its reliability through rigorous calibration with experienced air traffic controllers. Airspace complexity is defined as an essential indicator to comprehensively measure the safety of air traffic operational situations. A reliable prediction of airspace complexity can provide practical guidance for formulating air traffic management strategies and resource allocation. Although extensive efforts have been devoted to computing airspace complexity, previous studies can rarely model the multi-dimensional and combined spatio-temporal features within airspace complexity data. In this paper, we propose a multimodal adaptive spatio-temporal graph neural network to simultaneously explore the spatio-temporal dependencies in the airspace sector network. Specifically, we design a multimodal adaptive graph convolution module to effectively learn the diverse spatial relationships and adaptively adjust the impact of different spatial modes on airspace complexity in a data-driven manner. To model dynamic long-short-term temporal patterns, we develop a dilated causal convolution layer with a multiple-time-step self-attention mechanism to accurately predict airspace complexity over a longer time horizon. Extensive experiments on real-world air traffic datasets show that the proposed approach can harness differing spatial modes in achieving higher generalization performance across different temporal patterns, outperforming state-of-the-art methods in all prediction time horizons. [ABSTRACT FROM AUTHOR]

The fault management process is facilitated by equipping power distribution systems with automated devices, especially remote-controlled switches (RCSs). Although RCS plays a key role in improving the system's reliability, it imposes significant investment, installation, and maintenance costs. Hence, RCSs should be optimally located in distribution feeders for the highest profit. In previous works, reliability-oriented mathematical optimization models have been formulated to reach this goal. However, the number of test solutions exponentially grows with the problem size to find the globally optimal solution. This paper uses machine learning to propose a scalable and easy-to-implement model for optimal switch placement in real power distribution systems. At first, the features of candidate points for installing RCSs are introduced. Then, a learning model is applied to deeply explore the relationship between these features and optimal locations for installing RCSs. After training, the learning-based surrogate model directly determines the optimal RCS placement strategy in real power distribution systems by leveraging knowledge gained from past experiences. Simulation results demonstrate that the proposed surrogate model is approximately 29 times faster than the integer programming-based mathematical model, without a significant loss of accuracy, when implemented on a modified 11 kV network connected to Bus 4 of the Roy Billiton test system. This paper also employs explainable artificial intelligence (XAI) tools to select the most important features, where the Hamming loss is decreased by approximately 5%. [ABSTRACT FROM AUTHOR]

Obtaining the aerodynamic performance of the turbine blade by Computational Fluid Dynamics (CFD) methods is accurate. However, it consumes time and computational resources. This paper proposes an evaluation method based on Convolutional Neural Network (CNN) and Artificial Neural Network (ANN) to obtain the aerodynamic performance of the turbine blade accurately and quickly. Compared with the existing data-driven modeling methods, this method innovatively introduces the Residual Network (ResNet), employs a transfer learning strategy for network design, and realizes the automatic extraction of blade profile features and non-parametric input. In processing boundary conditions, the ANN is utilized to fuse the blade profile features with the boundary conditions to realize the mapping between blade profile and aerodynamic performance under different conditions. In addition, to minimize the prediction deviation caused by the severely uneven distribution of the data set, we combined ensemble learning with transfer learning and proposed a two-step prediction strategy. The numerical simulations results show that the ResNet-ANN model established in this paper has a prediction relative error of 5 % on turbine blade aerodynamic parameters under various working conditions. The error is reduced by more than 90 % under −40°-10° incidence angle of incoming flow compared with the empirical model. • The method for predicting turbine blade profile loss is non-parametric. • The scheme is effective on both design and off-design conditions. • A two-step strategy is proposed for precise prediction on unevenly distributed data. • The TSP strategy is based on transfer learning and ensemble learning. • The approach is of high accuracy with considerable generalization. [ABSTRACT FROM AUTHOR]

The aging of transformer oil–paper insulation (OPI) contributes significantly to transformer failure and reduced service life. Therefore, reliable and precise evaluation of the aging status of transformer insulation condition is crucial in averting transformer failure. This article proposes a novel deep learning-based aging condition assessment technique of transformers by combining FDS (Frequency domain spectroscopy) and squeeze-and-excitation-enabled convolutional neural network (SE-CNN). An intrinsic product-based transformation using a Gramian angular field is employed to dimensionally expand the 1-D FDS signals acquired from OPI samples into 2D images. Each feature extraction channel in the SE-CNN-based aging diagnosis framework is assigned a specific weight, enabling the model to concentrate on the most relevant features while discarding the rest. Furthermore, the effect of moisture variation on the model is also considered under three different conditions to ensure the consequence of compensation for the effect of moisture on FDS. A comparison with traditional machine learning models has also been made to ensure the viability of the proposed methodology. Additionally, the model is interpreted by creating explanation weight plots of the feature coefficients using a local interpretable model-agnostic explanations (LIME) method in Python. The findings demonstrate that the novel aging diagnosis framework visually assesses the transformer's OPI system with the highest accuracy of 99.12% in the test set. Furthermore, it facilitates the automatic extraction of deep aging features, thus providing better generalization and convergence capability. [ABSTRACT FROM AUTHOR]

[Display omitted] • A corn quality detection method based on spectral technology and deep learning was proposed. • This method solves the time-consuming and labor-intensive problem of corn quality detection. • This paper provides a new idea and method for corn quality detection. • The research results provide technical support for the rapid detection of crop quality. Corn is an important food crop in the world. With economic development and population growth, the nutritional quality of corn is of great significance to high-quality breeding, scientific cultivation and fine management. Aiming at the problems of cumbersome steps, time-consuming and laborious, and low accuracy in the current research on corn quality detection. This paper proposes to combine near-infrared (NIR) spectroscopy technology with deep learning technology to build a corn quality detection model based on convolutional neural network (LeNet-5). The original spectral data were preprocessed by wavelet transform (WT) and multivariate scattering correction (MSC) to remove noise interference and spectral scattering information. The Competitive Adaptive Reweighted Sampling Algorithm (CARS) was applied to optimize the characteristic wavenumber and reduce redundant data. According to the optimized characteristic wave number, it was input into the constructed corn quality detection model for simulation test, and the average detection accuracy rate of the test set was 96.46%, the average precision rate was 95.42%, the average recall rate was 97.92%, the average F1score was 96.64%, and the average recognition time was 51.95 s. Compared with traditional machine learning models such as BP neural network, K Nearest Neighbor (KNN), Support Vector Machine (SVM), Generalized Linear Model (GLM), Linear Discriminant Analysis (LDA), and Naive Bayesian (NB), the deep learning LeNet-5 network model constructed in this paper has an average accuracy increase of 39.32%, and has a higher detection accuracy. [ABSTRACT FROM AUTHOR]

The global production of concrete, being one of the most commonly utilised materials in the world, is associated with significant CO 2 -emissions and a substantial depletion of mineral resources. In order to improve sustainability, concretes thus are increasingly produced using recipes containing a large variety of different raw materials, including e.g. CO 2 reduced cements or recycled materials and industrial wastes. However, these actions result in heightened susceptibility of the concrete to variations in raw material characteristics, consequently diminishing the concrete's resilience and decreasing the concrete's robustness. Against this background, the quality control of fresh concrete before casting becomes of significant importance. However, current quality control is mainly conducted based on analogous, empirical, and often subjective test approaches. In this paper, a novel method is introduced for automatically and comprehensively evaluating the quality of fresh concrete at construction sites. In particular, the paper presents an end-to-end trainable framework for the image-based characterisation of fresh concrete properties. More specifically, a camera-based setup is proposed, in which image sequences of the discharge process of a mixing truck are acquired, on the basis of which the fresh concrete properties such as the consistency and rheology are determined. In this context, this paper introduces the concept of Spatio-Temporal Flow Fields , which provide a compact representation of the concrete's flow behaviour and serve as input to a multi-task convolutional neural network (CNN) predicting the target parameters describing the fresh concrete's properties. A thorough examination of the performance of the suggested method is carried out using highly challenging real-world data, showcasing extremely compelling outcomes with average prediction errors for the concrete properties of only about 6%. [Display omitted] • Novel strategy for camera-based characterisation of fresh concrete from flow observations. • Proposal of Spatio-Temporal Flow Fields as compact representation of the open-channel concrete flow. • A multi-task CNN predicting both, the fresh concrete's consistency and rheology • Highly promising results on a challenging real-world data set. [ABSTRACT FROM AUTHOR]

• Thousands of road-side images were collected in the manner of VGI. This paper contributes a VGI dataset with 20% rough labels and 80% accurate labels. • To handle the issue of crop type recognition in these images, a hierarchal semantic segmentation structure is proposed. Inspired by causality theories, the first stage of the model employs a binary classification to segment crops from background. The second stage aims to recognize the crop types including rape, corn, fallow and bare land, wheat and rice. As wheat and rice have many similar characteristics, they are considered as one category in this stage and will be further processed in the downstream. In the third stage, a priori knowledge such as the location and time of image collection is used for fine recognition between wheat and rice. • This structure can improve four semantic segmentation models' performance, among which Deeplabv3+ improves the most, with a mean IoU increase of 9 %. A method that directly adds prior information to the neural network to help similar crops' recognition is developed. Adding a priori information can also improve the performance with the mean precision, recall and IoU increased by 14%, 9% and 18%, respectively. The application of artificial intelligence in the field of agricultural crop type recognition is of great significance. There are currently two methods for collecting crop information: remote sensing and volunteered geographic information (VGI). As one pixel in a remote sensed satellite image may correspond to multiple crop types due to its low resolution, more refined VGI images can be a supplementary data source for agricultural crop monitoring. A hierarchal semantic segmentation structure based on Deeplabv3+ is proposed in this paper. Inspired by causality theories, a binary classification is adopted in the first stage of the model to segment crops from background. The second stage aims to recognize the crop types including rape, corn, fallow and bare land, wheat and rice. As wheat and rice have many similar characteristics, they are considered as one category in this stage and will be further processed in the downstream. In the third stage, a priori knowledge such as the location and time of image acquisition is utilized for fine recognition between wheat and rice. We have carried out experiments on the proposed Deeplabv3+ model and other different semantic segmentation models are validated on a VGI dataset with 20% rough labels and 80% accurate labels, including thousands of road-side images collected from different image sensors, under different weather and from different geographic locations. Experiments results show that Deeplabv3+ achieves the best performance. The recognition of five categories of crops has reached a precision of 87%, a recall of 91%, and IoU of 81%, showing good potential for use in the crop type recognition for VGI images. [ABSTRACT FROM AUTHOR]

This paper proposes a real-time state of health (SOH) estimation model based on a deep learning (DL) framework. The proposed model is a combination of two different architectures; a one-dimensional convolutional neural network (1D-CNN) and a bidirectional gated recurrent unit (BiGRU). The hybrid CNN-BiGRU uses the 1D CNN layers to extract pertinent features from input data and then relies on the BiGRU layers for sequence learning in both directions. To account for all SOH indicators, the proposed approach uses the current, voltage, and temperature measurements, which are readily obtainable from the electric vehicle's battery management system (BMS). This prevents the complex and time-consuming feature extraction used in most related papers. Since the hyperparameters have a significant impact on the performance of neural network models, a Bayesian Optimization (BO) technique based on the Gaussian Process (GP) was considered to tune the CNN-BiGRU model hyperparameters. Accordingly, the objective function was able to converge to a low Mean Squared Error (MSE) of 1.2 × 10 − 5 in just 19 iterations. Afterward, to verify the accuracy of the optimized model, a Lithium-ion battery dataset with several discharge profiles provided by the National Aeronautics and Space Administration (NASA) was used. The obtained results demonstrated the accuracy and robustness of the proposed method compared to other commonly used models. The CNN-BiGRU model yielded a Mean Absolute Error (MAE) of 2.080% and a root-mean-square error (RMSE) of 2.516% in the case of the battery set #C, referring to a set with 70 cycles already used at 24 °C. Additionally, the End of life (EOL) indicator error of zero cycles for the same data. [ABSTRACT FROM AUTHOR]

• This paper presents a memory augmented convolutional neural network. • This paper utilizes the augmenting SOM as memory for better network generalization. • This paper instantiates the distributed knowledge representation with SOM and CNN. The long short-term memory (LSTM) network underpins many achievements and breakthroughs especially in natural language processing fields. Essentially, it is endowed with certain memory capabilities to boost its performance. Currently, the volume and speed of big data generation are increasing exponentially, and such data require efficient models to acquire memory augmented knowledge. In this paper, we propose a memory augmented convolutional neural network (MACNN) with utilizing self-organizing maps (SOM) as the memory module. First, we depict the potential challenge about just applying solely a convolutional neural network (CNN) so as to highlight the advantage of augmenting SOM memory for better network generalization. Then, we dissert a corresponding network architecture incorporating memory to instantiate the distributed knowledge representation machanism, which tactically combines both SOM and CNN. Each component of the input vector is connected with a neuron in a two-dimensional lattice. Finally, we test the proposed network on various datasets and the experimental results reveal that MACNN can achieve competitive performance, especially for bioimages datasets. Meanwhile, we further illustrate the learned representations to interpret the SOM behavior and to comprehend the achieved results, which indicates that the proposed memory-incorporating model can exhibit the better performance. [ABSTRACT FROM AUTHOR]

MVI is a risk assessment factor related to hepatocellular carcinoma (HCC) recurrence after hepatectomy or liver transplantation. The goal of this paper is to study the preoperative diagnosis of microvascular invasion (MVI) by using a deep learning algorithm in non-contrast T2 weighted magnetic resonance imaging (MRI) images instead of pathological images. Herein, an ensemble learning algorithm named H-DARnet—based on the difference degree and attention mechanism, combined with radiomics, for MVI prediction—is proposed. Our hybrid network combines the fine-grained, high-level semantic, and radiomics features and exhibits a rich multilevel-feature architecture composed of global-local-prior knowledge with suitable complementarity. The total loss function comprises two regularization items––the triplet and the cross-entropy loss function––which are selected for the triplet network and SE-DenseNet, respectively. The hard triplet sample selection strategy for a triplet network and data augmentation for small-scale liver image datasets in convolutional neural network (CNN) training is indispensable. For 200 patch level test samples (135 positive samples and 65 negative samples), our method can obtain the best prediction results, the AUC, sensitivity, and specificity were 0.826, 79.5% and 73.8%, respectively. The experiment results show that MVI can be predicted by using MRI images, and the proposed method is better than other deep learning algorithms and hand-crafted feature algorithms. The proposed ensemble learning algorithm is proved to be an effective method for MVI prediction. • The goal of this paper is to study the preoperative diagnosis of HCC MVI by using a deep learning algorithm on T2WI. • An algorithm named H-DARnet based on the difference degree and attention mechanism, combined with radiomics is proposed. • Our hybrid network exhibits a rich multilevel-feature architecture composed of global-local-prior knowledge. • Hard triplet sample selection strategy and data augmentation for small-scale datasets in CNN training is indispensable. • For 200 patch level test samples, our method can obtain the AUC, sensitivity, and specificity 0.826, 79.5% and 73.8%. [ABSTRACT FROM AUTHOR]

To develop an efficient brain–computer interface (BCI) system, electroencephalography (EEG) measures neuronal activities in different brain regions through electrodes. Many EEG-based motor imagery (MI) studies do not make full use of brain network topology. In this paper, a deep learning framework based on a modified graph convolution neural network (M-GCN) is proposed, in which temporal-frequency processing is performed on the data through modified S-transform (MST) to improve the decoding performance of original EEG signals in different types of MI recognition. MST can be matched with the spatial position relationship of the electrodes. This method fusions multiple features in the temporal-frequency-spatial domain to further improve the recognition performance. By detecting the brain function characteristics of each specific rhythm, EEG generated by imaginary movement can be effectively analyzed to obtain the subjects' intention. Finally, the EEG signals of patients with spinal cord injury (SCI) are used to establish a correlation matrix containing EEG channel information, the M-GCN is employed to decode relation features. The proposed M-GCN framework has better performance than other existing methods. The accuracy of classifying and identifying MI tasks through the M-GCN method can reach 87.456%. After 10-fold cross-validation, the average accuracy rate is 87.442%, which verifies the reliability and stability of the proposed algorithm. Furthermore, the method provides effective rehabilitation training for patients with SCI to partially restore motor function. [ABSTRACT FROM AUTHOR]