Search

Timely and effective fault diagnosis is crucial to guarantee the reliability and durability of proton exchange membrane fuel cell (PEMFC). Segmented cell technology has been proven to be a powerful tool for exploring the internal characteristics of fuel cells, yet it is seldom applied for fault diagnosis. This paper proposes a data-driven fault diagnosis method that combines segmented cell and deep learning technology for PEMFC water management fault diagnosis. The dynamic characteristics of the current distribution of the fuel cell under different levels of faults are explored with segmented cell technology. A dual-input convolutional neural network is applied to combine current distribution data and sensor data for a more comprehensive system status for fault diagnosis. The results show that the model can efficiently identify flooding and drying faults with more than 98.5% accuracy, faster and more accurately than other algorithms. This work highlights the significance of segmented cell technology for the comprehension of water management inside PEMFC and provides a new concept for PEMFC fault diagnosis. [Display omitted] • The segmented cell technology is applied to investigate the current distribution during flooding and drying process. • A dual-input CNN is adopted to establish a fault diagnosis model for PEMFC water management. • The proposed method exhibits excellent diagnostic performance, with an accuracy rate of over 98.5%. [ABSTRACT FROM AUTHOR]

The large number of near-Earth asteroids (NEAs) has greatly impacted human space activities and Earth security. However, detecting NEAs in astronomical images with complex, varying backgrounds is still extremely challenging. In this paper, we propose a deep segmentation assisted asteroid tracking algorithm, termed DSAT, to construct a possible pipeline for faint NEA tracking in astronomical images. First, the single-frame object detection problem is converted to a segmentation problem, enabling robust extraction of faint potential moving objects. Then, a multiframe motion prior-based moving object tracking algorithm is proposed to find real NEAs. We further propose a distance tolerance criterion to help DSAT achieve effective tracking in practical situations when detection has partially failed. Finally, the pipeline is tested with both simulated and real astronomical images at different SNRs and in crowded fields. The results showed that our pipeline has the potential to detect and track faint NEAs in complex backgrounds. Our code is publicly available at https://github.com/zhenhongdu/DeepSegAsteroidTracker. [ABSTRACT FROM AUTHOR]

• The deep learning model can better extract the deeper features of the multi-point deformation sequence. • MVMD algorithm reduces the complexity of multi-point displacement sequence. • The model considers the spatiotemporal characteristics of dam deformation, the prediction accuracy is obviously improved. It is difficult for single measuring point model to synchronously characterize the spatial correlation of deformation sequence of concrete dam and the similarity of displacements of different measuring points. Considering the limitation of conventional model in dealing with spatial-temporal data structure, In this study, a deep learning technique for multi-point and multi-output deformation prediction of concrete dams is proposed, which is based on Multivariate Variational Mode Decomposition (MVMD) and three-dimensional convolutional neural network (3D-CNN) combined model. Firstly, the multi-point sequence is decomposed into several relatively stable sub-sequences with different frequency scales by MVMD, which reduces the complexity of the multi-point displacement sequence. Then, In order to enhance the model 's ability to fit the time and spatial features of the sequence, the 3D-CNN model is used to predict the decomposed components and reconstruct the displacement data of the measuring points. This study focuses on the principle, process and calculation steps of the proposed model, and verifies the proposed model through the deformation data of three typical measuring points. The test results show that the prediction model proposed in this paper is an effective method for dam deformation prediction. [ABSTRACT FROM AUTHOR]

Estimating the remaining useful life (RUL) of proton exchange membrane fuel cells (PEMFCs) is key to predicting and increasing their durability. This paper investigates six deep learning techniques – long short-term memory (LSTM) networks, gated recurrent units (GRU), 1-D convolutional neural network (CNN)-LSTMs, 1D–CNN–GRUs, 1D–CNN–Bidirectional-LSTMs, and 1D–CNN–Bidirectional-GRUs – for RUL and long-term degradation trend prediction for a five-cell PEMFC stack operated with and without current ripples. The hyperparameters of all models are optimized for minimum root-mean-square error (RMSE) of the stack voltage using tree-structured Parzen estimators (TPE) with hyperband pruning. The obtained prediction results are benchmarked using the coefficient of determination (R2), RMSE, and mean absolute percentage error (MAPE) in stack voltage predictions and relative error (RE) in RUL estimates. The comparative analysis for models trained on 50% datasets reveals that the best R2, RMSE, MAPE, and RE of 98.87E-2, 1.258E-3, 0.285E-1, and 0.0425E-1 are offered by the LSTM model for the stack without current ripples. Further, for the stack with current ripples, the best R2, RMSE, MAPE, and RE of 99.482E-2, 1.77E-3, 0.444E-1, and 0.089E-1 are exhibited by the 1D–CNN–Bidirectional-GRU model. The LSTM model also reveals excellent transfer learning performance on the stack with current ripples with the best R2, RMSE, MAPE, and RE of 98.91E-2, 2.455E-3, 0.362E-1, and 0.044E-1, respectively. [Display omitted] • Six deep learning models are developed for PEMFC RUL and degradation prediction. • Hyperparameters of each model are optimized using TPE for static and dynamic loading. • Static loading models exhibited excellent transfer learning performance. • All tuned models performed superior to recent reported approaches in RMSE. • LSTM and 1D–CNN–Bidirectional-GRU demonstrated best results trained on 50% data. [ABSTRACT FROM AUTHOR]

Objective: Tissue mechanical properties are valuable markers for tissue characterization, aiding in the detection and staging of pathologies. Shear wave elastography (SWE) offers a quantitative assessment of tissue mechanical characteristics based on the SW propagation profile, which is derived from the SW particle motion. Improving the signal-to-noise ratio (SNR) of the SW particle motion would directly enhance the accuracy of the material property estimates such as elasticity or viscosity. Methods: In this paper, we present a 3-D multi-resolution convolutional neural network (MRCNN) to perform improved estimation of the SW particle velocity V z. Additionally, we propose a novel approach to generate training data from real acquisitions, providing high SNR ground truth target data, one-to-one paired to inputs that are corrupted with real-world noise and disturbances. Discussion: By testing the network on in vitro data acquired from a commercial breast elastography phantom, we show that the MRCNN outperforms Loupas' autocorrelation algorithm with an improved SNR of 4.47 dB for the V z signals, a two-fold decrease in the standard deviation of the downstream elasticity estimates, and a two-fold increase in the contrast-to-noise ratio of the elasticity maps. The generalizability of the network was further demonstrated with a set of ex vivo porcine liver data. Conclusion: The proposed MRCNN outperforms the standard autocorrelation method, in particular in low SNR regimes. [ABSTRACT FROM AUTHOR]

In personal identity recognition systems, detecting a person's age, gender, and language using voice signal characteristics is a crucial issue, especially because of the importance of security considerations. Age, gender, and language classification problems are important in signal processing because they are used to analyze and understand human behavior, interactions, and preferences. This can be especially useful in the fields of human-computer interaction, psychology, and social science research. In this paper, a new system for detecting a speaker's age, gender, and language based on deep learning models is presented. Deep learning models have shown great efficacy in various fields of signal processing. For this paper, a range of deep models were tested, including convolutional neural networks (CNNs), recurrent neural network (RNN), and a fine-tuning ResNet34 architecture. Additionally, techniques such as transfer learning were applied to improve the efficiency of the proposed system. The input voice signals are preprocessed by applying the spectro-temporal transform to obtain additional features that can be fed to the ResNet34 model, which is designed specifically for the task of voice signal processing. The dataset used in this paper was sourced from the Mozilla common voice initiative, which is dedicated to advancing speech recognition and language identification technologies. The performance of the proposed algorithm was evaluated in the presence of Gaussian noise to determine its robustness. The experimental results demonstrated that the proposed algorithm significantly outperformed basic algorithms and other deep neural networks in terms of age and gender recognition from voice signals. [ABSTRACT FROM AUTHOR]

• We proposed a delineating field parcel model (TSANet) based on satellite image time series. • TSANet learn the relevance of spatial-spectral-temporal feature representation. • TSANet is robust across space and time. • TSANet performs better than pervious methods. • The time series data from the main growing period has a greater impact. Satellite image time series (SITS), such as Sentinel-2 imagery, plays a crucial role in the delineation of agricultural fields by reducing the impacts of ambiguities due to the spatial arrangement of field boundaries. Existing delineate field parcel models rely extensively on spatial features derived from single-date imagery. However, several studies have exploited the potential of SITS to effectively tackle the complexities associated with the intrinsic consistency between agricultural fields and their boundaries. This paper proposes a novel Two-Stream Attention convolutional neural network (TSANet) to capture the subtle difference between agricultural fields from SITS. Specifically, a field temporal semantic stream is introduced to adaptively leverage the significance of spatial-spectral-temporal feature representation associated with the location of agricultural parcels, especially where transitions in crop types take place. Considering the consistency between field parcels and their boundaries, we developed a field boundary prediction stream to enhance the extraction of edge features, particularly for the extraction of small and irregular agricultural parcels. Moreover, a field parcel refining block is employed to further enhance the geometric accuracy of agricultural fields. We conducted experiments on Sentinel-2 images from the Netherlands. Results showed that our approach produced a better layout of agricultural fields, with an average F1-score of 0.91 than the existing 3D-UNet, U-TEA, and BiConvLSTM. In addition, through the analysis of both quantitative and qualitative results, the stronger robustness of the model compared to other algorithms has been verified by temporal transfer and large-scale spatial prediction. We compared the difference between SITS and the corresponding composite images, which further verified the influence of temporal variation on the proposed approach. This paper provides a general guide for delineating agricultural parcels using SITS. [ABSTRACT FROM AUTHOR]

Food safety is an important public health issue, and deep learning (DL) algorithms can provide powerful tools and methods for food safety and authenticity detection. Compared with chemometric algorithms and traditional machine learning algorithms, the performances of DL algorithms are improved in many aspects. By learning and analyzing a large amount of data, DL models can improve the efficiency and accuracy of food safety and authenticity detection, helping to ensure the public health and safety. This paper reviews some commonly used chemometric algorithms, traditional machine learning algorithms, and popular DL algorithms. Among them, special attentions are paid to convolutional neural network (CNN), fully convolutional network (FCN) and generative adversarial network (GAN). Moreover, the auxiliary effect of GAN on CNN is highlighted. Finally, this paper revisits recent applications of DL algorithms in the field of food safety and authenticity detection, and prospects the challenges and future directions of DL algorithms in this field. Although DL has made many achievements in the field of food safety and authenticity detection, there is still a great potential for development. For example, the data augmentation function of GAN can assist CNN to obtain more training samples, thus improving the recognition rate. In addition, multimodal neural network (MNN) or multimodal attention network (MAN) can be also used to achieve the fusion of data from different modalities to further improve the robustness and accuracy of DL algorithms. [Display omitted] • Some popular DL algorithms in food safety and authenticity detection are introduced. • The improvements of DL algorithms to chemometric and ML algorithms are summarized. • Recent applications of DL algorithms in food safety and authenticity detection are reviewed. • Future prospects of DL algorithms in food safety and authenticity detection are discussed. [ABSTRACT FROM AUTHOR]

Because the carbon load inside a diesel particulate filters (DPF) affects the DPF regeneration, and the carbon load recognition is significant for the particulate matter (PM) emission control. It is necessary to investigate an on-board DPF carbon load recognition method because the carbon load cannot be directly measured by sensors. Aiming to build a DPF carbon load prediction model adopting the deep learning method, this paper proposes a DPF carbon load identification model based on different experimental parameters using a layered one dimension convolutional neural network (1D-CNN) method. To improve data validity, this paper adopts two data-processing methods. The data pre-processing adopts data splicing method to complete the construction of the original sample set, and the data after-processing uses wavelet packet transform method to establish the feature sample sets. The model adopts the optimal feature dataset constructed by three input parameters, i.e., temperature difference, pressure difference, and exhaust mass flow, and has both high training accuracy and test accuracy above 90 %. The pressure difference is the most important influencing input parameter, and the three-parameter sample set (Δ T + Δ P + Q) has great recognition accuracy and good model stability with the high training accuracy and test accuracy as well as less iteration. • A heavy-duty diesel engine was tested under various conditions of carbon load inside DPF. • The data processing was done by two methods: data splicing and wavelet packet transform. • We proposed a DPF carbon load identification model based on a one-dimension CNN method. • The model adopts 3 parameters, and the 3-parameter sample set has optimal performance. • The 1D-CNN model has both high training accuracy and test accuracy above 90 %. [ABSTRACT FROM AUTHOR]

Groundwater level (GWL) prediction is important for ecological protection and resource utilization; it helps in formulating policies for artificial groundwater recharge, modifying the number of extraction wells, etc., and can support sustainable human development as well as inform water resource management decisions. However, climate change, anthropogenic impacts, and the complex coupling between surface water and groundwater increase the difficulty of predicting groundwater levels. The model proposed in this paper combines external data as well as multiple models. The method leverages long and short-term memory (LSTM) and convolutional neural network (CNN) models, combined with secondary modal decomposition and slime mould algorithm (SMA), together with an adaptive weight module (AWM). The study applies this method to predict GWL for three different hydrological conditions in China, specifically for the Jinan Baotu Spring, Heihu Spring, and Zhongtianshe watershed of Taihu Lake. A comparison of metrics such as mean absolute error and Nash efficiency coefficient for single and hybrid models shows that the model in this paper is more advantageous than the single model and other hybrid models. The interpretability of the model is enhanced by SHAP values that demonstrate the degree of contribution of the input variables. This paper uses SHAP analyses to identify the key drivers affecting groundwater levels. These factors must be detected in order to develop groundwater resource protection measures. [Display omitted] • Multivariate fusion data including hydrology and meteorology are used as model input. • A secondary modal decomposition module for historical groundwater level data was utilized. • The neural network hyperparameters are optimized using the slime mould algorithm. • Aggregate subsets of prediction with adaptive modules rather than linear summation. • The interpretable SHAP model measures the degree of influence of external variables. [ABSTRACT FROM AUTHOR]

A tidal turbine can benefit from exquisitely designed morphing blades with a flexible trailing edge by mitigating up to 90% of the load fluctuation in harsh ocean environments, which reduces the overall cost of tidal energy. However, existing fluid–structure interaction (FSI) methods of resolving flow-induced deformation of the blades is computationally expensive, which poses an important challenge to effective morphing blade design. This paper presents a novel static FSI tool based on deep learning to cost-efficiently analyze the fluid–structure coupling of a hydrofoil. Specifically, adopting a convolutional neural network (CNN) to predict the fluid force and finite element method (FEM) to solve the solid structure response, a new CNN-FEM framework with an iterative scheme for solving the FSI problem is developed to achieve equilibrium between the fluid and structural forces. The new framework is used to predict the elastic deformation of the flexible blade section of the hydrofoil to demonstrate its effectiveness in the FSI evaluation. Comparison of the results to those produced by commercially developed software (i.e., Ansys Workbench) shows that this method yields extremely close prediction results of average equivalent stress and an accuracy of more than 92%. Moreover, it is 100 times more computationally efficient than the commercial Ansys Workbench software, requiring less than 3s for one-way FSI calculation. Taking advantage of this cost-effectiveness, the CNN-FEM can be used to achieve the accurate prediction of the deformation characteristics of the flexible hydrofoil under various flow scenarios that lay a foundation for advanced morphing blade design in the future. [ABSTRACT FROM AUTHOR]

In recent years, deep learning (DL) methods have proven their efficiency for various computer vision (CV) tasks such as image classification, natural language processing, and object detection. However, training a DL model is expensive in terms of both complexities of the network structure and the amount of labeled data needed. In addition, the imbalance among available labeled data for different classes of interest may also adversely affect the model accuracy. This paper addresses these issues using a new convolutional neural network (CNN) based architecture. The proposed network incorporates both spatial and spectral information that combines two sub-networks: spatial-CNN and spectral-CNN. The spectral-CNN extracts spectral information, while spatial-CNN captures spatial information. Moreover, to make the features more robust, a multiscale spatial CNN architecture is introduced using different kernels. The final feature vector is formed by concatenating the outputs obtained from both spatial-CNN and spectral-CNN. To address the data imbalance problem, a generative adversarial network (GAN) was used to generate data for the underrepresented class. Finally, relatively a shallower network architecture was used to reduce the number of parameters in the network and improve the processing speed. The proposed model was trained and tested on Senitel-2 images for the classification of the debris-covered glacier. The results showed that the proposed method is well-suited for mapping and monitoring debris-covered glaciers at a large scale with high classification accuracy. In addition, we compared the proposed method with conventional machine learning approaches, support vector machine (SVM), random forest (RF) and multilayer perceptron (MLP). [ABSTRACT FROM AUTHOR]

Modern and new integrated technologies have changed the traditional systems by using more advanced machine learning, artificial intelligence methods, new generation standards, and smart and intelligent devices. The new integrated networks like the Internet of Things (IoT) and 5G standards offer various benefits and services. However, these networks have suffered from multiple object detection, localization, and classification issues. Conventional Neural Networks (CNN) and their variants have been adopted for object detection, classification, and localization in IoT networks to create autonomous devices to make decisions and perform tasks without human intervention and helpful to learn in-depth features. Motivated by these facts, this paper investigates existing object detection and recognition techniques by using CNN models used in IoT networks. This paper presents a Conventional Neural Networks for 5G-Enabled Internet of Things Network (CNN-5GIoT) model for moving and static objects in IoT networks after a detailed comparison. The proposed model is evaluated with existing models to check the accuracy of real-time tracking. The proposed model is more efficient for real-time object detection and recognition than conventional methods. [ABSTRACT FROM AUTHOR]

The paper proposes a new class of nonlinear operators and a dual learning paradigm where optimization jointly concerns both linear convolutional weights and the parameters of these nonlinear operators. The nonlinear class proposed to perform a rich functional representation is composed by functions called rectified parametric sigmoid units. This class is constructed to benefit from the advantages of both sigmoid and rectified linear unit functions, while rejecting their respective drawbacks. Moreover, the analytic form of this new neural class involves scale, shift and shape parameters to obtain a wide range of activation shapes, including the standard rectified linear unit as a limit case. Parameters of this neural transfer class are considered as learnable for the sake of discovering the complex shapes that can contribute to solving machine learning issues. Performance achieved by the joint learning of convolutional and rectified parametric sigmoid learnable parameters are shown to be outstanding in both shallow and deep learning frameworks. This class opens new prospects with respect to machine learning in the sense that main learnable parameters are attached not only to linear transformations, but also to a wide range of nonlinear operators. [ABSTRACT FROM AUTHOR]

The fault management process is facilitated by equipping power distribution systems with automated devices, especially remote-controlled switches (RCSs). Although RCS plays a key role in improving the system's reliability, it imposes significant investment, installation, and maintenance costs. Hence, RCSs should be optimally located in distribution feeders for the highest profit. In previous works, reliability-oriented mathematical optimization models have been formulated to reach this goal. However, the number of test solutions exponentially grows with the problem size to find the globally optimal solution. This paper uses machine learning to propose a scalable and easy-to-implement model for optimal switch placement in real power distribution systems. At first, the features of candidate points for installing RCSs are introduced. Then, a learning model is applied to deeply explore the relationship between these features and optimal locations for installing RCSs. After training, the learning-based surrogate model directly determines the optimal RCS placement strategy in real power distribution systems by leveraging knowledge gained from past experiences. Simulation results demonstrate that the proposed surrogate model is approximately 29 times faster than the integer programming-based mathematical model, without a significant loss of accuracy, when implemented on a modified 11 kV network connected to Bus 4 of the Roy Billiton test system. This paper also employs explainable artificial intelligence (XAI) tools to select the most important features, where the Hamming loss is decreased by approximately 5%. [ABSTRACT FROM AUTHOR]

Convolutional Neural Network is one of the famous members of the deep learning family of neural network architectures, which is used for many purposes, including image classification. In spite of the wide adoption, such networks are known to be highly tuned to the training data (samples representing a particular problem), and they are poorly reusable to address new problems. One way to change this would be, in addition to trainable weights, to apply trainable parameters of the mathematical functions, which simulate various neural computations within such networks. In this way, we may distinguish between the narrowly focused task-specific parameters (weights) and more generic capability-specific parameters. In this paper, we suggest a couple of flexible mathematical functions (Generalized Lehmer Mean and Generalized Power Mean) with trainable parameters to replace some fixed operations (such as ordinary arithmetic mean or simple weighted aggregation), which are traditionally used within various components of a convolutional neural network architecture. We named the overall architecture with such an update as a hyper-flexible convolutional neural network. We provide mathematical justification of various components of such architecture and experimentally show that it performs better than the traditional one, including better robustness regarding the adversarial perturbations of testing data. [ABSTRACT FROM AUTHOR]

Shadows not only reduce image quality but also interfere with image interpretation, and accurate shadow extraction is the key to improving remote sensing image utilization. However, complex features lead to shadow extraction difficulties in remote sensing imagery. In this paper, an omni-scale global–local aware network (OGLANet) is proposed by analyzing the typical characteristics of shadows in remote sensing images. First, we establish a global–local aware module (GLAM) for fully extracting shadow features to solve the problem regarding the insufficient ability to control global and local network features. Second, the detailed and semantic information of shadows exists on different scales. We propose a dense feature fusion module (DFFM) between the encoder and decoder so that the detailed information can be restored in the decoding stage while retaining the semantic information. Finally, to solve the extreme scale differences of shadows, an omni-scale aggregation module (OAM) is established; this module can obtain more refined results in the prediction stage. To prove the effectiveness of our method, we compare it with state-of-the-art (SOTA) deep learning models proposed in recent studies on the same dataset. The results show that our method achieves higher accuracy and that the proposed OGLANet exhibits higher robustness and transferability than other methods. [ABSTRACT FROM AUTHOR]

Time series(TS) forecasting has been widely applied in many fields and industrial soft sensor is one of them. Most time series modeling methods require that all inputs are sampled at equal intervals. However, in industry, the process variables are often sampled on different time scales with varying intervals. To address this problem, in this paper, we designed a framework using deep learning with time representation techniques to model the long temporal industrial data with multiple sampling frequency. Data are aggregated into different time scale by time representation and the network extracts the information on both temporal and spatial dimensions simultaneously using the bottleneck layer and one-dimensional filter. The proposed model has a significant improvement compared with other methods and has been deployed in the factory and updated every month. [ABSTRACT FROM AUTHOR]

Deep Learning (DL) has been recently employed to build smart systems that perform incredibly well in a wide range of tasks, such as image recognition, machine translation, and self-driving cars. In several fields the considerable improvement in the computing hardware and the increasing need for big data analytics has boosted DL work. In recent years physiological signal processing has strongly benefited from deep learning. In general, there is an exponential increase in the number of studies concerning the processing of electromyographic (EMG) signals using DL methods. This phenomenon is mostly explained by the current limitation of myoelectric controlled prostheses as well as the recent release of large EMG recording datasets, e.g. Ninapro. Such a growing trend has inspired us to seek and review recent papers focusing on processing EMG signals using DL methods. Referring to the Scopus database, a systematic literature search of papers published between January 2014 and March 2019 was carried out, and sixty-five papers were chosen for review after a full text analysis. The bibliometric research revealed that the reviewed papers can be grouped in four main categories according to the final application of the EMG signal analysis: Hand Gesture Classification, Speech and Emotion Classification, Sleep Stage Classification and Other Applications. The review process also confirmed the increasing trend in terms of published papers, the number of papers published in 2018 is indeed four times the amount of papers published the year before. As expected, most of the analyzed papers (≈ 60 %) concern the identification of hand gestures, thus supporting our hypothesis. Finally, it is worth reporting that the convolutional neural network (CNN) is the most used topology among the several involved DL architectures, in fact, the sixty percent approximately of the reviewed articles consider a CNN. [ABSTRACT FROM AUTHOR]

[Display omitted] • Proposal of a large-scale dataset for active fire detection in Landsat-8 imagery. • Deep learning can approximate and improve the results from handcrafted algorithms. • Deep learning networks can approximate combinations of handcrafted algorithms. Active fire detection in satellite imagery is of critical importance to the management of environmental conservation policies, supporting decision-making and law enforcement. This is a well established field, with many techniques being proposed over the years, usually based on pixel or region-level comparisons involving sensor-specific thresholds and neighborhood statistics. In this paper, we address the problem of active fire detection using deep learning techniques. In recent years, deep learning techniques have been enjoying an enormous success in many fields, but their use for active fire detection is relatively new, with open questions and demand for datasets and architectures for evaluation. This paper addresses these issues by introducing a new large-scale dataset for active fire detection, with over 150,000 image patches (more than 200 GB of data) extracted from Landsat-8 images captured around the world in August and September 2020, containing wildfires in several locations. The dataset was split in two parts, and contains 10-band spectral images with associated outputs, produced by three well known handcrafted algorithms for active fire detection in the first part, and manually annotated masks in the second part. We also present a study on how different convolutional neural network architectures can be used to approximate these handcrafted algorithms, and how models trained on automatically segmented patches can be combined to achieve better performance than the original algorithms – with the best combination having 87.2% precision and 92.4% recall on our manually annotated dataset. The proposed dataset, source codes and trained models are available on Github (https://github.com/pereira-gha/activefire), creating opportunities for further advances in the field. [ABSTRACT FROM AUTHOR]

Recently, the successful applications of convolutional neural network (CNN) in computer vision have attracted considerable attentions. Particularly, deep learning models with attention mechanisms have shown impressive performance in hyperspectral (HS) pansharpening. However, most of these existing models follow an early/late-fusion strategy and do not take full advantage of the hierarchical features. In this paper, we specifically design a novel end-to-end multi-level feature fusion network with cross-layer guided attention for HS pansharpening, termed as HP-MFFN, which allows the network to extract the hierarchical features level by level. Specifically, as the different levels of the network have different receptive fields and contain different details, the hierarchical features extracted from the HS image and panchromatic (PAN) image are refined by the cross-layer guided attention fusion module (called CLGAF) to yield more effective spatial-spectral features with fine details and rich semantics. The experimental results conducted on widely-used datasets demonstrate that HP-MFFN provides high-quality pansharpened HS images in terms of perceptually and quantitatively. [ABSTRACT FROM AUTHOR]

• We review artificial intelligence applications of brain image preprocessing. • We investigate several commonly seen brain diseases and survey their intelligence-based methods. • We discuss AI's future in brain disease studies. Brain diseases threaten hundreds of thousands of people over the world. Medical imaging techniques such as MRI and CT are employed for various brain disease studies. As artificial intelligence succeeded in image analysis, scientists employed artificial intelligence, especially deep learning technologies, to assist brain disease studies. The AI applications for brain disease studies can be divided into two categories. The first category is preprocessing, including denoising, registration, skull-stripping, intensity normalization, and data augmentation. The second category is the clinical application that contains lesion segmentation, disease detection, grade classification, and outcome prediction. In this survey, we reviewed over one hundred representative papers on how to apply AI to brain disease studies. We first introduced AI-based preprocessing for brain disease studies. Second, we reviewed the influential works of AI-based brain disease studies. At last, we also discussed three development trends in the future. We hope this survey will inspire both expert-level researchers and entry-level beginners. [ABSTRACT FROM AUTHOR]

The evaluation of node importance is a critical research topic in network science, widely applied in social networks, transport systems, and computer networks. Prior works addressing this topic either consider a single metric or assign weights for multiple metrics or select features by handcraft, which exist one-sidedness and subjectivity issues. In this paper, to tackle these problems, we propose a new approach named CGNN to identify influential nodes based on deep learning methods, including Convolutional Neural Networks (CNNs) and Graph Neural Networks (GNNs). CGNN obtains the feature matrices by the contraction algorithm and gets the labels by the Susceptible-Infected-Recovered (SIR) model, which will be leveraged for learning the hidden representations of nodes without utilizing any network metrics as features. We adopt three evaluation criteria to verify CGNN concerning effectiveness and distinguishability, including Kendall's τ correlation coefficient, monotonicity index (MI), and ranking distribution function (RDF). Nine baselines are employed to compare with CGNN on thirty synthetic networks and twelve real-world networks from different domains. Simulation results demonstrate that CGNN manifests better performance than the baselines, in which the values of τ are large and significantly increase, the values of MI approach to 1, and the points in the RDF curves distribute more uniformly. These results may provide reference significance for controlling epidemic spreading and enhancing network robustness. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have significantly advanced computational modelling for saliency prediction. However, accurately simulating the mechanisms of visual attention in the human cortex remains an academic challenge. It is critical to integrate properties of human vision into the design of CNN architectures, leading to perceptually more relevant saliency prediction. Due to the inherent inductive biases of CNN architectures, there is a lack of sufficient long-range contextual encoding capacity. This hinders CNN-based saliency models from capturing properties that emulate viewing behaviour of humans. Transformers have shown great potential in encoding long-range information by leveraging the self-attention mechanism. In this paper, we propose a novel saliency model that integrates transformer components to CNNs to capture the long-range contextual visual information. Experimental results show that the transformers provide added value to saliency prediction, enhancing its perceptual relevance in the performance. Our proposed saliency model using transformers has achieved superior results on public benchmarks and competitions for saliency prediction models. The source code of our proposed saliency model TranSalNet is available at: https://github.com/LJOVO/TranSalNet. [ABSTRACT FROM AUTHOR]

With the prevalence and the enormous societal consequence on health of Alzheimer's disease (AD), diagnosis of AD and its prodromal form, mild cognitive impairment (MCI) is essential for patient care, and has been a research hotspot in recent years. Existing studies have applied machine learning methods to perform AD early diagnosis by analyzing various biomarkers. However, the difficulty in extracting the low-dimensional high-level brain features that accurately reflect main AD-related variations of anatomical brain structures becomes a bottleneck of the diagnosis performance in most of the existing researches. To overcome this bottleneck, this paper proposes a novel three-component adversarial network-based AD detection method (brain slice generative adversarial network for Alzheimer's disease detection, BSGAN-ADD) to predict the disease category. BSGAN-ADD combines generative adversarial network (GAN)-based brain slice image enhancement and deep convolutional neural network (CNN)-based AD detection. In BSGAN-ADD, under the restriction of the discriminator, the generator learns to integrate the disease category feedbacks from classifier into 2D-brain slice image reconstruction process for image enhancement in the training phase. In the prediction phase, the stacked CNN layers in the generator are used to extract high-level brain features from category-enhanced 2D-brain slice images. And the classifier receives the extracted brain features to output the posterior probabilities of diseased states (Normal, AD and MCI). Experimental results on two real-world datasets (Alzheimer's disease neuroimaging initiative, ANDI, Open Access Series of Imaging Studies OASIS) demonstrate that the new feature extraction process used in BSGAN-ADD can extract more representative high-level brain features to achieve a significant diagnosis performance gain compared with several typical methods. [ABSTRACT FROM AUTHOR]

Alternating current optimal power flow (OPF) analysis is critical for efficient and reliable operation of power systems. For large systems or repetitive computations, the traditional methods such as the direct and gradient methods, or non-traditional methods, such as the genetic algorithm and simulating annealing, are time-consuming and unsuitable for real-time computing. The work in this paper proposes a novel framework to obtain the optimal solution of power flow in real-time using a combination of convolutional neural networks and a self-attention mechanism. All parameters of the power networks are rearranged in an image-like shape of a multi-channel image where each channel is a two-dimensional matrix. The proposed approach is adaptive with every input size of power systems as well as frequent variations of network topologies without intervention to the framework core. The encompassment of all power system contexts in which all parameters of internal elements, generation costs, and topology information are included, contributes to the higher accuracy of inference compared to other current machine-learning-based OPF-solving methods. Besides, the proposed framework established on ubiquitous platforms is effortlessly integrated into current infrastructures of power systems, and the great efficiency along with the computation speed may serve as a critical point for practical implications, such as enabling faster decision-making during real-time operations, predicting system contingencies, and remedial actions based on an offline pre-trained model. This supervised learning process is applied to the dataset of four case studies of meshed power systems: the IEEE 5-bus system (IEEE-5), the IEEE 30-bus system (IEEE-30), the IEEE 39-bus system (IEEE-39), and the IEEE 57-bus system (IEEE-57) to prove the efficacy of the proposed method. • Developing a novel approach to solve the OPF problems in real-time. • Wrap power system characteristics under a mold of an image-like shape. • Overcome the outburst of trainable parameters when training with large-scale systems. • Do not require looking up a new set of hyperparameters or re-coding a framework. • All time-consuming computations are shifted to the offline training process. [ABSTRACT FROM AUTHOR]

Fruit ripeness estimation models have for decades depended on spectral index features or colour-based features, such as mean, standard deviation, skewness, colour moments, and/or histograms for learning traits of fruit ripeness. Recently, few studies have explored the use of deep learning techniques to extract features from images of fruits with visible ripeness cues. However, the blackberry (Rubus fruticosus) fruit does not show obvious and reliable visible traits of ripeness when mature and therefore poses great difficulty to fruit pickers. The mature blackberry, to the human eye, is black before, during, and post-ripening. To address this problem, this paper proposes a novel multi-input convolutional neural network (CNN) ensemble classifier (MCE) for detecting subtle traits of ripeness in blackberry fruits. The multi-input CNN was created from a pre-trained visual geometry group 16-layer deep convolutional network (VGG16) model trained on the ImageNet dataset. The fully connected layers were optimized for learning traits of ripeness of mature blackberry fruits. The resulting model of about 600 K trainable parameters served as the base for building homogeneous ensemble learners t ensembled using the stack generalization ensemble (SGE) framework. The input to the network are images acquired with a stereo sensor using visible and near-infrared (Vis-NIR) spectral filters at wavelengths of 700 nm and 770 nm. Through experiments, the proposed model achieved 95.1 % accuracy on unseen sets and 90.2 % accuracy with in-field conditions. Further experiments reveal that machine sensory is highly and positively correlated to human sensory over blackberry fruit skin texture • Fruits that do not show obvious and reliable visible traits of ripeness when mature pose great difficulty in fruit ripeness estimation. • Hyperspectral imaging was investigated for evaluating blackberry fruit ripeness and determining the spectral wavelengths that optimally correspond to ripeness in mature blackberry fruit. • A novel multi-input convolutional neural network (CNN) ensemble classifier was proposed for detecting subtle traits of ripeness in images of blackberry fruits. • The proposed model with blackberry fruit images captured at spectral wavelengths of 700 nm and 770 nm achieved an 11.3 % and 16.2 % increase in test accuracies compared to the RGB-based model. • Bi-spectral imagery significantly improves the ripeness-sensing capabilities of deep learning techniques. • Machine sensory has a ripeness sensing capability that is highly and positively correlated to human sensory over blackberry fruit skin texture. [ABSTRACT FROM AUTHOR]

Implementing key engineering solutions to optimise the operation of energy industries requires daily electricity demand forecasting and including uncertainty, to promote markets insight analysis as part of their strategic planning, regulating and supplying electricity to consumers. This paper proposes hybrid artificial intelligence models combining convolutional neural networks (CNN) as a feature extraction algorithm with extreme learning machines (ELM) as a framework to predict electricity demand with confidence intervals generated by Kernel Density Estimation (KDE) approaches. In order to develop CELM-KDE model, time-lagged series of daily electricity demand with local climate variables based on the air temperature, atmospheric vapour pressure, evaporation, solar radiation, humidity and sea level pressures are used to train the proposed CELM-KDE hybrid model. In order to fully evaluate the newly developed model from a point-based, as well as a probabilistic prediction strategy, the observed and predicted electricity demand as well as the probability distribution of errors are analysed using KDE method that operates without prior data distribution assumptions. Based on observed and predicted electricity demand and the relevant probabilistic confidence intervals generated by the CELM-KDE model, the final results show that the proposed method attains significantly better probability interval predictions than traditionally-used point-based models. The proposed CELM-KDE model is demonstrated to be highly effective in providing a comprehensive coverage of predicted errors, as well as providing greater insights into the average bandwidth and detailed predicted electricity demand in the testing stage. The results also indicate that the proposed hybrid model is a reliable decision support tool to develop engineering solutions in area of energy modelling, monitoring and forecasting, which could potentially be useful to the industry policymakers. We show that the point-and probabilistic-based electricity demand predictive models can be employed as an effective tool to improve accuracy of forecasting and provision of insights for national electricity markets and key energy industry stakeholder application tools. [Display omitted] • A hybrid deep learning model for daily electricity demand prediction is proposed. • A Kernel Density Estimation is applied to derive probabilistic prediction intervals. • The algorithm quantifies uncertainties in point-based electricity demand predictions. • The proposed model outperforms deep learning models using statistical uncertainty indicators. [ABSTRACT FROM AUTHOR]

• A novel methodology integrating percussion and CNN model is established to detect concrete voids. • The CNN model trained by both CWT and MFCC show satisfying categorizing performances. • The methodology proves its good noise-immune capacity in processing percussion sound signals. • The superiority of the proposed method is well demonstrated by comparison with ML methods. As a fundamental construction material, concrete commonly encounter void defects, which could significantly impair the integrity of concrete members and seriously weaken their load-bearing capacity. Worse still, void defects typically exist inside concrete members, making it difficult to detect them in real time. Therefore, conducting regular inspections for internal void defects of concrete structures is of great importance to maintain structural health. This paper investigates an integrated method to detect concrete voids based on percussion approach and a convolutional neural network (CNN). Firstly, percussion-induced sound signals were collected from 18 concrete specimens with different precast void defects. Each specimen was tapped 100 times, for a total of 1,800 percussion-induced acoustic signals. Subsequently, continuous wavelet transform (CWT) and Mel frequency cepstral coefficient (MFCC) were introduced to convert the collected acoustic signals into two types of images. The converted images were divided into testing and training datasets according to the ratio of 1:3, and 600 images were input to the CNN as the test dataset and 1200 images as the training dataset. The CNN was then trained and verified by the joint input of CWT and MFCC images to perform concrete voids classification. The result shows that the proposed method presents a satisfying classification accuracy of 100 %. Next, percussion tests under 5 levels of noisy environments were conducted, and the results prove the remarkable anti-noise performance of the methodology by achieving 88.12 % accuracy even at the strongest signal-to-noise ratio of 0 dB. Finally, the superiority of the proposed method was further demonstrated through the accuracy comparison with machine learning (ML) methods, with the accuracy consistently outperforming over 25 %. In conclusion, the proposed method has a great potential for future in-situ applications in concrete void detection. [ABSTRACT FROM AUTHOR]

Deep learning models often rely on a diverse and well-augmented dataset for optimal performance. In this context, the methods of data augmentation are pivotal in boosting the models' ability to generalize. In this paper, we introduce a novel data augmentation method, which we call RGB-Angle-Wheel, to improve the performance of deep learning models on RGB format images. This method involves rotating each color channel at specific angles to generate new training data that is distinct from the original dataset but shares similar properties. Experimental results on the CIFAR-10, CIFAR-100,and COCO datasets have validated the efficacy of the proposed method for enhancing model performance. Specifically, certain transformations in the red (R) and blue (B) channels improve model accuracy significantly, whereas the effect on the green (G) channel remains limited. These results indicate that the careful selection of transformation parameters plays a critical role in enhancing model performance. The findings of the study indicate that the proposed method can be utilized specifically for image processing, image classification, object detection, and other deep learning applications. Experiments demonstrate that the proposed method improves the model's efficacy and generalizability. [ABSTRACT FROM AUTHOR]

Many IoT devices are presently in use without sufficient security measures. The vulnerability of these devices to malware highlights the necessity for effective methods to identify malicious activity while protecting data privacy. Federated Learning is a collaborative learning technique that can create a fair model while keeping the local data private. To this end, this paper presents the SIM-FED, a novel model for detecting malware in IoT devices. This model utilizes both deep learning and federated learning algorithms. The SIM-FED offers a privacy-preserving solution for IoT malware detection, without sharing data, enhancing security in distributed environments. The model utilizes a lightweight one-dimensional CNN with optimized hyperparameters, reducing preprocessing, and computational overhead. Various federated aggregation strategies are evaluated, and the FedAvg strategy is selected to integrate the results of local models in the proposed model. Moreover, the model's resilience against white-box and black box cyber-attacks is assessed, revealing minimal performance degradation and highlighting its robustness, which is often overlooked in previous studies. The performance of the SIM-FED is assessed by conducting a series of experiments using the IoT-23 dataset. The results show that the SIM-FED outperforms other deep learning and federated learning models in terms of all evaluation metrics, achieving a remarkable 99.52 % accuracy. [ABSTRACT FROM AUTHOR]

[Display omitted] • The first time using CBAM-CNN to predict particle size distribution during fluidized bed granulation. • The CBAM-CNN model was optimized by introducing Bayesian optimization and C-Mixup algorithms. • The performance of CBAM-CNN was compared to other classic models. • The modeling convenience of CBAM-CNN was evaluated. Monitoring the particle size distribution (PSD) is crucial for controlling product quality during fluidized bed granulation. This paper proposed a rapid analytical method that quantifies the D10, D50, and D90 values using a Convolutional Block Attention Module-Convolutional Neural Network (CBAM-CNN) framework tailored for deep learning with near-infrared (NIR) spectroscopy. This innovative framework, which fuses CBAM with CNN, excels at extracting intricate features while prioritizing crucial ones, thereby facilitating the creation of a robust multi-output regression model. To expand the training dataset, we incorporated the C-Mixup algorithm, ensuring that the deep learning model was trained comprehensively. Additionally, the Bayesian optimization algorithm was introduced to optimize the hyperparameters, improving the prediction performance of the deep learning model. Compared with the commonly used Partial Least Squares (PLS), Support Vector Machine (SVM), and Artificial Neural Network (ANN) models, the CBAM-CNN model yielded higher prediction accuracy. Furthermore, the CBAM-CNN model avoided spectral preprocessing, preserved the spectral information to the maximum extent, and returned multiple predicted values at one time without degrading the prediction accuracy. Therefore, the CBAM-CNN model showed better prediction performance and modeling convenience for analyzing PSD values in fluidized bed granulation. [ABSTRACT FROM AUTHOR]

Development of human machine interface has become a necessity for modern day machines to catalyze more autonomy and more efficiency. Gaze driven human intervention is an effective and convenient option for creating an interface to alleviate human errors. Facial landmark detection is very crucial for designing a robust gaze detection system. Regression based methods capacitate good spatial localization of the landmarks corresponding to different parts of the faces. But there are still scope of improvements which have been addressed by incorporating attention. In this paper, we have proposed a deep coarse-to-fine architecture called LocalEyenet for localization of only the eye regions that can be trained end-to-end. The model architecture, build on stacked hourglass backbone, learns the self-attention in feature maps which aids in preserving global as well as local spatial dependencies in face image. We have incorporated deep layer aggregation in each hourglass to minimize the loss of attention over the depth of architecture. Our model shows good generalization ability in cross-dataset evaluation and in real-time localization of eyes. • Deep architecture build on stacked hourglass backbone for localization of eye regions. • Self-attention module between hourglasses to learn local spatial and global attention. • Designed a deep layer aggregation module to learn feature dependencies across network. • Differentiable soft-argmax to make the framework an end-to-end trainable architecture. • Precise and fast localization of eyes tested in real-time with varied illumination. [ABSTRACT FROM AUTHOR]

Accurate fault identification provides an important basis for well location deployment, oil and gas resource development. However, obtaining a large number of fault samples through manual labeling or interpretation is difficult. Conventional convolutional neural networks have certain challenges in finely characterizing low-order faults and fault continuity. This paper studies a 3D intelligent fault identification method that integrates deep learning with attention mechanism. Based on 3D-UNet network, the method constructs a dual attention fault identification model with an encoder-decoder structure, which focuses on enhancing low-order fault information and extracting key features. An adaptive hybrid loss function is proposed to address the imbalance problem of fault proportions in seismic amplitude data. The function introduces an adjustment coefficient to adaptively enhance the attention to high-order or low-order faults according to the actual situation of the fault, improving the accuracy and continuity of fault identification. Furthermore, a model re-training learning optimization method is presented to capture the missed fault information in the first training and repair the connectivity of low-order faults. Applying this method to the synthetic data set, the accuracy can be improved to 0.9732, and the loss can be converged to 0.0982. The results of shale reservoirs in the North Sea Basin and Kerry-3D indicate that the method can reduce the situation of fault missing and misidentification, effectively improve fault identification accuracy, and enhance low-order fault identification by obtaining more contextual fault features. • A seismic fault image segmentation network is developed with attention mechanism. • Construct an adaptive hybrid loss function to solve the problem of data imbalance. • A method retraining learning that integrates theory and predictive data is proposed. [ABSTRACT FROM AUTHOR]

The detection and classification of concrete damage is essential for maintaining good infrastructure condition. Traditional semantic segmentation methods often can not provide accurate crack boundary information, which limits the further location and measurement analysis. In this study, the case segmentation method is used to solve the shortcomings of the previous detection methods and achieve more accurate crack identification results. This paper presents an improved YOLOv7 network design scheme. The network includes three different custom modules that can optimize the algorithm to solve missing feature problems, small recognition frames, and gradient problems, thereby improving accuracy. In addition, data sets with different sizes, exposures and noise are used to train the network, which expands the prediction range of the network and enhances the stability of the network. The experimental results show that compared with YOLOv7, YOLOv5, SOLOv2, Cascade Mask R-CNN, Condinst, Sparseinst, mAP is significantly improved. Thus, the proposed network algorithm has high practical engineering value. • Crack morphology detection and category classification are achieved using instance segmentation. • Two innovative modules and a unique connection are developed. • A dataset for crack with high exposure, darkness, and noise is constructed. • The model achieves mAP of 72.2% and 87.9% on public and self-built datasets. [ABSTRACT FROM AUTHOR]

[Display omitted] Empirical formulas, established by experimental statistical methods, have been a traditional method for predicting permeability in porous materials, but they suffer from shortcomings in terms of prediction accuracy. In this paper, a framework conjugating generative adversarial nets (GAN) and convolutional neural networks (CNN) is proposed for the rapid prediction of permeability based on 3D Berea sandstone images. This framework aims to surpass the limitations of empirical formulas in achieving accurate predictions. The framework involves three steps: porous material images reconstruction based on a conditional porosity progressive growth of generative adversarial nets (CPPGGAN), permeability label calculation based on the Lattice Boltzmann method (LBM) and deep learning prediction model construction using convolutional neural network (CNN). The CNN predictor trained on 3920 CPPGGAN synthetic images achieved an R2 value of 0.927 and 0.925 on 560 synthetic images and 60 original images, respectively. This indicated the effectiveness of the CPPGGAN synthetic images in the permeability prediction framework. The framework can also be adapted to a wider range of porous material types and shows great potential for exploring materials with exceptional properties in nature. [ABSTRACT FROM AUTHOR]

• Brute force detection and face recognition models are established. • This model can be applied to actual public security defense. • The CNP model based on SPP proposed in this paper belongs to the improved "cross space method". • This method maintains high input mode in multi-scale and maintains high sampling accuracy. With the emergence of the concept of "safe city", security construction has gradually been valued by various cities, and video surveillance technology has also been continuously developed and applied. However, as the functional requirements of actual applications become more and more diverse, video surveillance systems also need to be more intelligent. The purpose of this article is to study methods of brute force detection and face recognition based on deep learning. Aiming at the problem of abnormal behavior detection, especially the low efficiency and low accuracy of brute force detection, a brute force detection method based on the combination of convolutional neural network and trajectory is proposed. This method uses artificial features and depth features to extract the spatiotemporal features of the video through a convolutional neural network and combines them with the trajectory features. In view of the problem that face images in surveillance video cannot be accurately recognized due to low resolution, two models are proposed: the multi-foot input CNN model and the SPP-based CNN model. By testing the performance of the brute force detection method proposed in this paper, the accuracy of the method on the Crow and Hockey datasets is as high as 92% and 97.6%, respectively. Experimental results show that the violence detection method proposed in this paper improves the accuracy of violence detection in video. [ABSTRACT FROM AUTHOR]

• Taking advantage of the fact that CatBoost does not require significant preprocessing of category features, the internal construction uses unique thermal coding for categorical features with small bases, enabling the model to automatically transform category-based features into numerical features. • The proposed model supports automatic finding of class feature combinations, which significantly extends the dimensionality of license plate hyperspectral image features by maximizing the relationship between different features, further improving the effectiveness and generalization ability of the model. • By comparing with traditional image processing methods, the proposed CatBoost-based license plate hyperspectral image classification model has better performance in processing applications. The rapid development of artificial intelligence technology has brought profound technical changes to many traditional industries, among which intelligent transportation has become a hot spot for development in the traditional transportation field. Remote sensing license plate image recognition technology is extensively emphasized in such domains as intelligent transportation and intelligent vehicle management. However, in the practical traffic environment, low visibility scenes caused by complex environmental factors such as rain, snow, haze and cloudy days influence the recognition and classification of license plates, while the distortion of license plate images that may be caused by irregular movements of vehicles bring challenges to license plate recognition classification. The CNN-CatBoost model proposed in this paper divides the license plate recognition classification into two stages. The first stage uses the excellent performance of convolutional neural network in processing image data to extract various license plate image features; the second stage uses the CatBoost module to further process the image feature data and finally obtain the remote sensing license plate information. The model achieves outstanding results in the experiments and has practical application value. Through comparison with other network models, the CNN-CatBoost model proposed in this paper has superior performance. [ABSTRACT FROM AUTHOR]

• This paper explores the potential of spatial feature dimensions to seek an effective feature representation for point clouds. • The dimension expansion of MKConv enables spatially correlated feature representation and discrete convolutions for point cloud data without information loss. • For stable and effective learning, kernel weight normalization and multidimensional attention are proposed. • The effectiveness of multidimensional feature representation is demonstrated by the superior performance on three different point cloud processing tasks including object classification, object part segmentation, and scene semantic segmentation. Despite the remarkable success of deep learning, an optimal convolution operation on point clouds remains elusive owing to their irregular data structure. Existing methods mainly focus on designing an effective continuous kernel function that can handle an arbitrary point in continuous space. Various approaches exhibiting high performance have been proposed, but we observe that the standard pointwise feature is represented by 1D channels and can become more informative when its representation involves additional spatial feature dimensions. In this paper, we present Multidimensional Kernel Convolution (MKConv), a novel convolution operator that learns to transform the point feature representation from a vector to a multidimensional matrix. Unlike standard point convolution, MKConv proceeds via two steps. (i) It first activates the spatial dimensions of local feature representation by exploiting multidimensional kernel weights. These spatially expanded features can represent their embedded information through spatial correlation as well as channel correlation in feature space, carrying more detailed local structure information. (ii) Then, discrete convolutions are applied to the multidimensional features which can be regarded as a grid-structured matrix. In this way, we can utilize the discrete convolutions for point cloud data without voxelization that suffers from information loss. Furthermore, we propose a spatial attention module, Multidimensional Local Attention (MLA), to provide comprehensive structure awareness within the local point set by reweighting the spatial feature dimensions. We demonstrate that MKConv has excellent applicability to point cloud processing tasks including object classification, object part segmentation, and scene semantic segmentation with superior results. [ABSTRACT FROM AUTHOR]

This paper proposes a robust algorithm using two convolutional neural networks (CNN) to find a lost gamma source in a room despite various unpredictable obstacles. This algorithm is an improved and complementary method to our previous research, which does not limit the use of the algorithm to a specific geometry, and the presence of obstacles does not reduce the performance and inability of the method. In our previous research, an algorithm including a CNN was used to estimate the location of the lost gamma source in known geometry. In this paper, by adding another CNN to the previous algorithm, the algorithm can be used in environments with unknown geometry. The environmental simulations and data generation are performed using the Geant4 simulation program. The method has been tested in a designed scenario to check its effectiveness. The results show that by using these two CNNs, an agent can intelligently find the gamma source place in a room with multiple obstacles without human intervention with 95 percent accuracy. [ABSTRACT FROM AUTHOR]

Object detection is one significant field of computer vision. The imbalance problem exerts negative effects on achieving satisfactory performance. We reveal two sources of imbalance in existing object detection methods. Correspondingly, we propose our methods in terms of the model architecture and optimization target. Different from general object detection benchmarks, the location distribution of objects with different sizes is unbalanced in many practical applications. In addition, the representation information of different categories of objects is unbalanced. In this paper, we propose a location scale equilibrium module to utilize the prior location scale information and generate more balanced feature maps. More appropriate feature maps are selected and merged for different locations. After merging, feature maps become more consistent in terms of representation content, exerting positive effects on the following classification and regression tasks. For the imbalance caused by similar objects, we propose the repulsive loss to strengthen the punishment. Our method will not treat all categories of objects equally since we take the imbalance between them into consideration. With the enhanced supervision, the training will pay more attention to similar objects. Our proposed model is evaluated on the VisDrone benchmark and UAVDT benchmark. Sufficient experiments are conducted. Our model achieves the highest precision on most evaluation metrics, outperforming the other strong models. [ABSTRACT FROM AUTHOR]

Over the last years, advances in deep learning have brought huge developments to the studying of super-resolution reconstruction. However, most super-resolution methods only deal with simply down-sampled sharp images, which may lose efficacy when encountering severe blur. The severe motion blur caused by the rapid movement of an object or the large shake of the lens is common in video captured by cameras. However, existing super-resolution algorithms often bring a large amount of artifacts and are difficult to achieve satisfactory results when reconstructing such blurred video sequences. In this paper, a novel convolutional neural network is proposed that jointly processes video super-resolution (SR) and deblurring (DB) to deal with severe motion blur and recover sharp high-resolution (HR) frames. In particular, a pyramid optical flow module is introduced to estimate the sharp latent image in the blurred frame and generate HR optical flow in a coarse-to-fine way. Then, the frame-recurrent is used to warp the previous SR frame to achieve motion compensation and make full use of the previous sharp features and temporal information to help restoration of the subsequent frames. Next, to further overcome the destruction caused by motion blur in the final reconstruction, a parallel-fusion module was designed to extract and fuse the SR and DB features, finally reconstructing the output frame. Experimental results obtained in this study confirm that, compared with other advanced SR algorithms, the proposed method is both effective and efficient in dealing with videos that contain real motion blur. [ABSTRACT FROM AUTHOR]

Super-resolution (SR) and image enhancement (IE) are two common and important tasks in image processing. Due to poor equipment, illumination, and photographic skills, many low-resolution and low-quality (in terms of color, and contrasts) photos are captured in our daily life. To make the photos more visually pleasing, we usually enlarge and retouch them in two separate steps. In this paper, we propose a new Joint Image Super-resolution and Enhancement Network (JSENet), which is arguably the first end-to-end method based on deep learning for joint SR and IE. Compared with simply conducting these two tasks in parallel or sequentially, JSENet employs the bilateral learning framework to integrate the two tasks together seamlessly. Meanwhile, two lightweight modules are designed for restoring details and generating color transformation coefficients respectively, which ensures that our JSENet can be deployed in real-world applications. Extensive experiments are conducted to demonstrate that JSENet outperforms the state-of-the-art SR-IE methods. [ABSTRACT FROM AUTHOR]

[Display omitted] Disparity estimation of satellite stereo images is an essential and challenging task in photogrammetry and remote sensing. Recent researches have greatly promoted the development of disparity estimation algorithms by using CNN (Convolutional Neural Networks) based deep learning techniques. However, it is still difficult to handle intractable regions that are mainly caused by occlusions, disparity discontinuities, texture-less areas, and repetitive patterns. Besides, the lack of training datasets for satellite stereo images remains another major issue that blocks the usage of CNN techniques due to the difficulty of obtaining ground-truth disparities. In this paper, we propose an end-to-end disparity learning model, termed hierarchical multi-scale matching network (HMSM-Net), for the disparity estimation of high-resolution satellite stereo images. First, multi-scale cost volumes are constructed by using pyramidal features that capture spatial information of multiple levels, which learn correspondences at multiple scales and enable HMSM-Net to be more robust in intractable regions. Second, stereo matching is executed in a hierarchical coarse-to-fine manner by applying supervision to each scale, which allows a lower scale to act as prior knowledge and guides a higher scale to attain finer matching results. Third, a refinement module that incorporates the intensity and gradient information of the input left image is designed to regress a detailed full-resolution disparity map for local structure preservation. For network training and testing, a dense stereo matching dataset is created and published by using GaoFen-7 satellite stereo images. Finally, the proposed network is evaluated on the Urban Semantic 3D and GaoFen-7 datasets. Experimental results demonstrate that HMSM-Net achieves superior accuracy compared with state-of-the-art methods, and the improvement on intractable regions is noteworthy. Additionally, results and comparisons of different methods on the GaoFen-7 dataset show that it can severs as a challenging benchmark for performance assessment of methods applied to disparity estimation of satellite stereo images. The source codes and evaluation dataset are made publicly available at https://github.com/Sheng029/HMSM-Net. [ABSTRACT FROM AUTHOR]

Some invasive tree species threaten biodiversity and cause irreversible damage to global ecosystems. The key to controlling and monitoring the propagation of invasive tree species is to detect their occurrence as early as possible. In this regard, one-class classification (OCC) shows potential in forest areas with abundant species richness since it only requires a few positive samples of the invasive tree species to be mapped, instead of all the species. However, the classical OCC method in remote sensing is heavily dependent on manually designed features, which have a limited ability in areas with complex species distributions. Deep learning based tree species classification methods mostly focus on multi-class classification, and there have been few studies of the deep OCC of tree species. In this paper, a deep positive and unlabeled learning based OCC framework— ITreeDet —is proposed for identifying the invasive tree species of Eucalyptus spp. (eucalyptus) and Acacia mearnsii (black wattle) in the Taita Hills of southern Kenya. In the ITreeDet framework, an absNegative risk estimator is designed to train a robust deep OCC model by fully using the massive unlabeled data. Compared with the state-of-the-art OCC methods, ITreeDet represents a great improvement in detection accuracy, and the F1-score was 0.86 and 0.70 for eucalyptus and black wattle, respectively. The study area covers 100 km2 of the Taita Hills, where, according to our findings, the total area of eucalyptus and black wattle is 1.61 km2 and 3.24 km2, respectively, which represent 6.78% and 13.65% of the area covered by trees and forest. In addition, both invasive tree species are located in the higher elevations, and the extensive spread of black wattle around the study area confirms its invasive tendency. The maps generated by the use of the proposed algorithm will help local government to develop management strategies for these two invasive species. [ABSTRACT FROM AUTHOR]

The Covid-19 pandemic is the defining global health crisis of our time. Chest X-Rays (CXR) have been an important imaging modality for assisting in the diagnosis and management of hospitalised Covid-19 patients. However, their interpretation is time intensive for radiologists. Accurate computer aided systems can facilitate early diagnosis of Covid-19 and effective triaging. In this paper, we propose a fuzzy logic based deep learning (DL) approach to differentiate between CXR images of patients with Covid-19 pneumonia and with interstitial pneumonias not related to Covid-19. The developed model here, referred to as CovNNet , is used to extract some relevant features from CXR images, combined with fuzzy images generated by a fuzzy edge detection algorithm. Experimental results show that using a combination of CXR and fuzzy features, within a deep learning approach by developing a deep network inputed to a Multilayer Perceptron (MLP), results in a higher classification performance (accuracy rate up to 81%), compared to benchmark deep learning approaches. The approach has been validated through additional datasets which are continously generated due to the spread of the virus and would help triage patients in acute settings. A permutation analysis is carried out, and a simple occlusion methodology for explaining decisions is also proposed. The proposed pipeline can be easily embedded into present clinical decision support systems. [ABSTRACT FROM AUTHOR]

Heart sound classification plays a vital role in the early detection of cardiovascular disorders, especially for small primary health care clinics. Despite that much progress has been made for heart sound classification in recent years, most of them are based on conventional segmented features and shallow structure based classifiers. These conventional acoustic representation and classification methods may be insufficient in characterizing heart sound, and generally suffer from a degraded performance due to the complicated and changeable cardiac acoustic environment. In this paper, we propose a new heart sound classification method based on improved Mel-frequency cepstrum coefficient (MFCC) features and convolutional recurrent neural networks. The Mel-frequency cepstrums are firstly calculated without dividing the heart sound signal. A new improved feature extraction scheme based on MFCC is proposed to elaborate the dynamic characteristics among consecutive heart sound signals. Finally, the MFCC-based features are fed to a deep convolutional and recurrent neural network (CRNN) for feature learning and later classification task. The proposed deep learning framework can take advantage of the encoded local characteristics extracted from the convolutional neural network (CNN) and the long-term dependencies captured by the recurrent neural network (RNN). Comprehensive studies on the performance of different network parameters and different network connection strategies are presented in this paper. Performance comparisons with state-of-the-art algorithms are given for discussions. Experiments show that, for the two-class classification problem (pathological or non-pathological), a classification accuracy of 98% has been achieved on the 2016 PhysioNet/CinC Challenge database. • We propose a new MFCC-based heart sound classification method. • A new improved MFCC feature extraction scheme is developed. • CNN and RNN are combined for heart sound classification. • We show promising performance on benchmark PCG databases. [ABSTRACT FROM AUTHOR]

This paper proposes an automatic visual inspection method for the fracture detection of clevises in the catenary systems of high-speed railways, using images of catenaries captured by an inspection vehicle. First, the clevises are extracted from the catenary image using a convolutional neural network based algorithm, known as the faster region-based convolutional neural network. Because the structure of catenary systems does not have many variations and the contextual information near a catenary fitting may have strong correlation with its category, the architecture of the original faster region-based convolutional neural network is modified to make use of the contextual information of the regions of interest in the images for object recognition. A crack detection process is then used to recognize the fractures of clevises. To detect the cracks, the edge map of the clevis sub-image is generated using a region-scalable fitting model. Areas where the cracks are most likely to occur are projected from a standard clevis image to the clevis sub-image by shape context matching and affine transformation matrix computation. The cracks are then recognized by calculating the wavelet entropy inside these areas followed by morphological filtering. Experimental results show that the modified faster region-based convolutional neural network architecture achieves better results in clevis extraction than the original architecture as well as some other state-of-art object detection models. The detection is not affected by the scaling, texture and grayscale changes of the clevises caused by the variation of shooting distance, shooting angle and illumination variations. The fractures of the clevises can be accurately and reliably detected using the fracture detection method proposed in this paper and the performance of this visual inspection method meets the strict requirements for catenary system maintenance. [ABSTRACT FROM AUTHOR]

Cervical spondylosis (CS) recognition systems provide regular screening services outside of a hospital and promote early detection and treatment of CS. However, in this paper, we propose a deep learning-based CS recognition system. Concerning the state-of-the-art and state-of-the-practice systems, the innovations of our approaches and algorithms are as follows: First, to elevate the reliance upon the sample number required for training the high-quality model, we reduce sample dimension and find optimal neural network architectures to reduce the number of model parameters to fit. Second, we incorporate multi-stream parallel network architecture search with multi-view feature extraction by converting time series classification into an image classification task. Specifically, five feature extraction methods (time-domain, frequency-domain, time–frequency domain, model-based, nonlinear feature extraction) are firstly utilized to extract features from multiple perspectives and form low-dimensional data set with multi-properties. Third, we reorganize low-dimensional data into image one representing the spatio-temporal relationship of muscle activity pattern. Finally, a multi-stream parallel network architecture search is proposed to use a bypass mechanism for optimal neural network architecture, each of which processes a kind of features mentioned above with an idea of the sparse connection of convolution neural network. The results on the real-world data set show that our CS recognition system achieves the average accuracy of 95.54%, average sensitivity of 99.09%, and average specificity of 90.00%, outperforming the state-of-the-art ones. [ABSTRACT FROM AUTHOR]