Search

Along with the growing popularity of deep learning to handle classification problems, various deep learning models have emerged having complex architectures. The disadvantage of a model with a complex architecture is the computational problem in which takes longer training time than a simple model. The desire to take advantage of sentiment classification in real-time applications is the reason for using a simpler model architecture but still paying attention to the model performance. Furthermore, this study also investigates how fastText embedding affects the performance of a sentiment classification model. In this paper, we propose two sentiment classification models with simple architecture. The first model is the single-layered Bidirectional Gated Recurrent Unit (BiGRU) model with fastText embedding, and the second one is the single-layered Convolutional Neural Network (CNN) model with fastText embedding. Both models can provide competitive results compared to the baseline model that has previously been compared with other models with complex architectures. The best accuracy is produced by the fastText + CNN model, with 80% of accuracy for the MR dataset and 84% of accuracy for the SST2 dataset. This study shows that the use of CNN for sentiment classification can provide competitive results compared to the BiLSTM and BiGRU models. This study also indicates that the use of fastText embedding can improve the performance of the single-layered BiLSTM model. [ABSTRACT FROM AUTHOR]

• We first exploit a deep framework with pattern theory to learn continuous temporal embedding. • We design a configuration proposal module to remove background information in video. • We explicitly formulate an incremental Viterbi algorithm for pattern theory. • The experimental results show that our CTE-PT overforms state-of-the-art methods. Visual Question Answering (VQA) is a challenging task in artificial intelligence and has received increasing attention from both the computer vision and the natural language processing communities. Joint embedding learning for it suffers from the background noise of images and text. Learning continuous temporal embedding is a potential solution to extract both visual elements and textual elements with a proper temporal structure. In this paper, we address a continuous temporal embedding model based on pattern theory (CTE-PT) and fully express the atomism and combinatory based on Grenander's pattern theory. First, we generate atomic actions from videos and note them as generators, which reflects the atomism in pattern theory. Second, we design a CTE-PT model to discover the discriminative configuration of videos, which reflects the combinatory of atomic actions in pattern theory. In the CTE-PT model, we design the configuration proposal module to remove some background information initially, and the configuration interpretation module to minimize the interpretive energy of continuous temporal embedding. We estimate the energy by each pair in this embedding sequence and optimize it based on the Viterbi algorithm. The experimental results show that our CTE-PT model outperforms the baseline C3D+LSTM model on Olympic Sports, UCF 101, and HMDB51 datasets, which proves the effectiveness of mining the common continuous temporal embedding as the class-specific configuration for activity discriminator. [Display omitted] [ABSTRACT FROM AUTHOR]

The novel coronavirus (COVID-19) could be described as the greatest human challenge of the 21st century. The development and transmission of the disease have increased mortality in all countries. Therefore, a rapid diagnosis of COVID-19 is necessary to treat and control the disease. In this paper, a new method for the automatic identification of pneumonia (including COVID-19) is presented using a proposed deep neural network. In the proposed method, the chest X-ray images are used to separate 2–4 classes in 7 different and functional scenarios according to healthy, viral, bacterial, and COVID-19 classes. In the proposed architecture, Generative Adversarial Networks (GANs) are used together with a fusion of the deep transfer learning and LSTM networks, without involving feature extraction/selection for classification of pneumonia. We have achieved more than 90% accuracy for all scenarios except one and also achieved 99% accuracy for separating COVID-19 from healthy group. We also compared our deep proposed network with other deep transfer learning networks (including Inception-ResNet V2, Inception V4, VGG16 and MobileNet) that have been recently widely used in pneumonia detection studies. The results based on the proposed network were very promising in terms of accuracy, precision, sensitivity, and specificity compared to the other deep transfer learning approaches. Depending on the high performance of the proposed method, it can be used during the treatment of patients. [ABSTRACT FROM AUTHOR]

Point-of-Interest (POI) recommendation systems can give the user a list of locations that the user may be interested in, but the recent check-ins may contain some daily check-ins that users are not really interested in. To address the issues of mining users' actual preferences from users' recent check-ins, this paper proposes a Convolutional Neural Networks with Attention (CNNA), which consists of a latent representation method and a deep convolutional neural network. We have carried out several sets of experiments to test the validity of CNNA, and the experimental results show the effectiveness of CNNA. [ABSTRACT FROM AUTHOR]

• Residual attention mechanisms adjust the flow of sentiment information. • The other aspect term module reduces the effect from other aspect terms' sentiment. • Experimental results show our model achieves state-of-the-art performance. Aspect-based sentiment analysis (ABSA) is a fine-grained sentiment analysis task designed to predict the sentiment polarity of each aspect term in a text. Recent research mainly uses neural networks to model text and utilizes attention mechanisms to interact for associate aspect terms and context to obtain more effective feature representation. However, the general attention mechanism is easy to lose the original information. Besides, in the multi-aspect text, the sentiment information of other aspect terms interferes with the sentiment analysis of the current aspect term likely. In this paper, we propose two models named RA-CNN and RAO-CNN for ABSA tasks. In RA-CNN, we apply CNN to model the aspect term and utilize a specially designed residual attention mechanism to interact with the text. Based on the RA-CNN, RAO-CNN adds other aspect terms module, which can reduce interference of sentiment information related to other aspect terms in the multi-aspect text. To verify the proposed models' effectiveness, we conduct a large number of experiments and comparisons on seven public datasets. Experimental results show that our proposed models are useful and achieve state-of-the-art results. [ABSTRACT FROM AUTHOR]

Multi-view action recognition (MVAR) is an optimal technique to acquire numerous clues from different views data for effective action recognition, however, it is not well explored yet. There exist several challenges to MVAR domain such as divergence in viewpoints, invisible regions, and different scales of appearance in each view require better solutions for real world applications. In this paper, we present a conflux long short-term memory (LSTMs) network to recognize actions from multi-view cameras. The proposed framework has four major steps; 1) frame level feature extraction, 2) its propagation through conflux LSTMs network for view self-reliant patterns learning, 3) view inter-reliant patterns learning and correlation computation, and 4) action classification. First, we extract deep features from a sequence of frames using a pre-trained VGG19 CNN model for each view. Second, we forward the extracted features to conflux LSTMs network to learn the view self-reliant patterns. In the next step, we compute the inter-view correlations using the pairwise dot product from output of the LSTMs network corresponding to different views to learn the view inter-reliant patterns. In the final step, we use flatten layers followed by SoftMax classifier for action recognition. Experimental results over benchmark datasets compared to state-of-the-art report an increase of 3% and 2% on northwestern-UCLA and MCAD datasets, respectively. [ABSTRACT FROM AUTHOR]

Increasing threats to U.S. national security satellite constellations have resulted in an increased interest in constellation resilience and satellite redundancy. CubeSats have contributed to commercial, scientific and government applications in remote sensing, communications, navigation and research and have the potential to enhance satellite constellation resilience. However, the inherent size, weight and power limitations of CubeSats enforce constraints on imaging hardware; the small lenses and short focal lengths result in imagery with low spatial resolution. Low resolution limits the utility of CubeSat images for military planning purposes and national intelligence applications. This paper implements a super-resolution deep learning architecture and proposes potential applications to CubeSat imagery. [ABSTRACT FROM AUTHOR]

• A spatio-temporal Siamese neural network is proposed for 3D signature recognition. • The system uses one shot learning to learn from small amounts of training data. • The proposed architecture outperforms conventional Siamese networks. • The ST-SNN outperforms current recognition techniques on raw 3D signature data. Signature forgery is at the centre of several fraudulent activities and legal battles. The introduction of 3D signatures, the virtual signing of ones name in the air, has the potential to restrict forgers due to the absence of visual cues that can be easily copied. Existing 3D signature recognition approaches, however, have not leveraged the inherent spatial and temporal information, making it difficult to handle the diminished separability and reproducibility of these signatures. In this paper, we propose a novel spatio-temporal adaptation of the Siamese Neural Network, wherein one branch extracts spatial features using a 1D Convolutional Neural Network (CNN) while the other processes the input in the temporal domain using Long Short-Term Memory networks (LSTMs). Unlike conventional deep learning networks, Siamese networks are an application of One-Shot Learning so as to learn from a small amount of data as is often the case in real life problems. They employ a distance metric that is forced to be small for like samples (signatures from the same person), and large for different samples (from different persons). The proposed approach, termed ST-SNN, is compared to other baseline classification architectures, and demonstrated using a publicly available biometric 3D signature benchmark dataset, yielding True Positive Rate (TPR) of 94.63% with 4.1% False Acceptance Rate (FAR). [ABSTRACT FROM AUTHOR]

A challenge for implementing deep learning research in the real-world is the availability of techniques that explain predictions of a model, particularly in light of potential legal requirements to give an account of algorithmic outcomes for certain use-cases. Convolutional neural networks are regularly proposed as an effective method of Android malware detection at scale. However there is a lack of studies into actually explaining the predictions of these classifiers, and how well they correlate with other state-of-the-art explainability methods. In this paper we present two research contributions that begin to address this issue. Firstly, a novel method to identify locations deemed important by our CNN in an Android app's opcode sequence which appear to contribute to malware detection. Secondly, a comparison of such locations highlighted by the CNN with those locations considered important from the state-of-the-art explainability method LIME. Using the publicly available Drebin benchmark dataset, our results show that for eight different malware families, when averaging location importance across all samples in each family, locations in an opcode sequence considered most malicious by our CNN match closely to those considered most malicious by LIME. This gives confidence the CNN is able to focus its attention on patterns of malicious opcodes in an Android app. As well as helping to validate use of CNNs for Android malware detection, our method may benefit the wider field of Android malware analysis to highlight opcode sequence locations worthy of investigation. [ABSTRACT FROM AUTHOR]

This research paper provides a framework for the efficient representation and analysis of both spatial and temporal dimensions of panel data. This is achieved by representing the data as spatio-temporal image-matrix, and applied to a case study on forecasting public transport ridership. The relative performance of a subset of machine learning techniques is examined, focusing on Convo-lutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) neural networks. Furthermore Sequential CNN-LSTM, Parallel CNN-LSTM, Augmented Sequential CNN-LSTM are explored. All models are benchmarked against a Fixed Effects Ordinary Least Squares regression. Historical ridership data has been provided in the framework of a project focusing on the impact that the opening of a new metro line had on ridership. Results show that the forecasts produced by the Sequential CNN-LSTM model performed best and suggest that the proposed framework could be utilised in applications requiring accurate modelling of demand for public transport. The described augmentation process of Sequential CNN-LSTM could be used to introduce exogenous variables into the model, potentially making the model more explainable and robust in real-life settings. [ABSTRACT FROM AUTHOR]

Animation drama and movies are important entertainment sources for children and young people. However, many of these entertainment videos introduce ideas such as violence, fighting, abuse, and car accidents, which are not suitable for children and young people. In order to prevent children from viewing these videos, different countries' government bodies put restrictions on age on these videos. Violence detection is mostly done using manual inspection, which is time and resource consuming and cannot achieve a very good result. Therefore, artificial technologies such as deep learning and machine learning can detect violence in a video. Bleeding is one of the distinct characteristics of violent actions presented in such videos. This paper aims to propose a system that can detect a given video's blood in real-time. Researchers have proposed different image processing-based techniques to detect violence, but their proposed systems lack precision and real-time detection. To this end, a deep learning-based approach is proposed to detect violence in a given video and images. A Faster R-CNN model is modified due to the sophisticated characteristics of the violence in the cartoon and animation images to achieve the highest performance. The backbone of the proposed violence detection model is changed to a modified RegNet model to extract the features from the frame. The standard inner lateral connection is replaced with the modulated deformable convolutional (MDC) layer to extract deformable feature maps. A novel distributed attention module (DAM) is proposed on the feature pyramid network to improve the performance of the feature extraction. The multiscale Region of interest (ROI) Align is adapted to improve the detection performance of violence in a different scenario. Moreover, the classification approach is integrated with the detection model to classify the different levels of violence in a given frame. We have explored state-of-the-art object detection methods such as Faster R-CNN, Cascade R-CNN, yolov3-spp, SSD, FCOS, and yolov5 to detect violence in a particular frame of a video. We compared the different models' performances and found out that the modified Faster R-CNN is appropriate for real-time blood detection in a given video or image with high accuracy. This study makes a significant contribution to detecting violence in animation videos which will help different platforms and government bodies to regulate entertainment content. [ABSTRACT FROM AUTHOR]

Anomaly detection is vital in complex distributed systems. However, existing methods did not take into full account the temporal and spatial characteristics of data from multiple sources in the system. That will lead to less accurate of anomaly detection. To address this limitation, in this paper we propose a multi-source data based anomaly detection method through temporal and spatial characteristics. Specifically, we first considered the spatial characteristics. We propose a Transformer encoder to parse log templates and output template vectors, then an attention-based CNN is introduced to obtain the spatial characteristics from traces. Next, we considered the temporal characteristics. We propose Bi-LSTM network to obtain the temporal characteristics of multi-source data of the distributed systems, followed by anomaly detection. Finally, extensive comparative experiments verify the effectiveness of our method, the F1-score reaches 0.859 and improves by 2.14% compared to state-of-the-art methods. [Display omitted] • Design a Transformer encoder to parse log templates and output template vectors. • Design an attention-based CNN to obtain the spatial characteristics from traces. • Use Bi-LSTM to obtain the temporal characteristics of logs, traces and metric data. • Detect anomalies by comparing the predicted data and the collected data. • Verify the superiority of our method in accuracy compared to SOTA methods. [ABSTRACT FROM AUTHOR]

Aiming at limitations in fully exploiting the temporal correlation features of the original signals, expensive cost in parameter tuning, and difficulties in obtaining sufficient training data under multiple working conditions, this paper proposes an optimized Convolutional Neural Network (CNN) with Bi-directional Long Short-Term Memory (BiLSTM) scheme for bearing fault diagnosis under multiple working conditions with limited training samples. A CNN-BiLSTM network is developed to obtain precise fault features and high detection accuracy by extracting high-dimensional and temporal correlation features of raw vibration signals. An improved particle swarm optimization (PSO) algorithm is leveraged to optimize the training hyperparameters of the CNN-BiLSTM network for further advances in fault diagnosis performance. The optimized CNN-BiLSTM network is regarded as a pre-trained model and transferred to new working conditions to achieve satisfactory fault diagnosis results based on limited training samples. Several comprehensive experiments are implemented to confirm the excellent performance of the proposed schemes, especially efficiently addressing the challenges of model training and fault diagnosis in new working conditions with scarce samples. [ABSTRACT FROM AUTHOR]

Smart grid is an organic combination of traditional power grid and advanced digital communication technology, and power communication network plays a vital role in ensuring its steady operation. However, as the coupling correlation between the smart grid and the power communication network increases, the propagation of coupled network failures may constitute a great threat to the safe and stable operation of the power system. To address this phenomenon, based on the algorithm of the multi-head attention mechanism (MHA) and convolutional neural network (CNN), this paper proposes a framework that integrates the multi-head attention mechanism with the convolutional neural network (MHA-CNN) for the monitoring of the coupling network's operation status. In the proposed algorithm, Hilbert–Huang transform (HHT) is used to extract the intrinsic features of the operational data, and MHA employs multi-subspace analysis and extracts the key feature information of the fault data to maximize the fault information extraction and help the system to discover the fault as soon as possible. Simulation results demonstrate that the proposed algorithm can effectively realize a better detection speed and higher detection probability compared with CNN and ResNet-50, and maintain certain robustness under poor transmission conditions, ensuring the steady operation of the coupled network. • Coupling of grid and power communication network state detection is essential. • A structure for state detection based on multi-attention mechanism is designed. • Multi-attention mechanism provides a multi-subspace representation of the data. • Efficient and precise state detection in coupled networks is realized. [ABSTRACT FROM AUTHOR]

Individuals in the early stages of Alzheimer's Disease (AD) are typically diagnosed with Mild Cognitive Impairment (MCI). MCI represents a transitional phase between normal cognitive function and AD. Electroencephalography (EEG) records carry valuable insights into cerebral cortex brain activities to analyze neuronal degeneration. To enhance the precision of dementia diagnosis, automatic and intelligent methods are required for the analysis and processing of EEG signals. This paper aims to address the challenges associated with MCI diagnosis by leveraging EEG signals and deep learning techniques. The analysis in this study focuses on processing the information embedded within the sequence of raw EEG time series data. EEG recordings are collected from 10 Healthy Controls (HC) and 10 MCI participants using 19 electrodes during a 30 min eyes-closed session. EEG time series are transformed into 2 separate formats of input tensors and applied to deep neural network architectures. Convolutional Neural Network (CNN) and ResNet from scratch are performed with 2D time series with different segment lengths. Furthermore, EEGNet and DeepConvNet architectures are utilized for 1D time series. ResNet demonstrates superior effectiveness in detecting MCI when compared to CNN architecture. Complete discrimination is achieved using EEGNet and DeepConvNet for noisy segments. ResNet has yielded a 3 % higher accuracy rate compared to CNN. None of the architectures in the literature have achieved 100 % accuracy except proposed EEGNet and DeepConvnet. Deep learning architectures hold great promise in enhancing the accuracy of early MCI detection. • EEG's high temporal resolution allows us to precisely track cognitive changes over time. • ResNet architecture is more effective than CNN for detecting dementia using time series data. • Sufficiently long segment length allows for the capture of discriminative patterns of MCI. • EEGNet model can efficiently capture and fuse temporal information at varying scales. • Preprocessing (filtering, normalization) steps may directly affect the performance of DL. [ABSTRACT FROM AUTHOR]

[Display omitted] • Expert decision system will prove beneficial for Glaucoma screening at large scale. • CNN based framework can be used as assistant by clinicians and healthcare practioners in limited facilities. • SLIC and normalized graph cut algorithm in conjoint gives good results in OD segmentation. Accurately segmenting the Optic Disk is a crucial step in classifying Glaucoma using Fundus images. Machine learning and artificial intelligence techniques are widely used in Glaucoma detection, and the main indicators observed in Fundus images are the presence of Papillary Atrophy, Cup to Disc Ratio values, diminishing Neural Retinal Rim (NRR), the Inferior Superior Nasal Temporal (ISNT) rule, and Cup Diameter. In this research, we investigated various segmentation and classification techniques that can be applied to Optic disk segmentation and classification of normal and glaucomatous eyes. The proposed method will be beneficial to clinicians and healthcare workers in facilities with limited resources. In this paper, histogram processing is used to determine the type of image, and based on this information; we decide whether the image requires segmentation. Some datasets in the standard dataset contain complete retinal images while others include segmented optic disks. The segmented images are directly given as input for classification using the proposed Convolutional Neural Network (CNN). For complete retinal images, segmentation is performed using the Simple Linear Iterative clustering (SLIC) and normalized graph cut algorithm. The proposed framework's performance is compared with that of pretrained neural networks, including VGG19, InceptionV3, and ResNet50V2, using major metrics. We trained and tested these architectures with 3115 images from six standard datasets. Our proposed framework outperforms all with an accuracy of 96.33 %. [ABSTRACT FROM AUTHOR]

• A novel double U-shaped network that combines CNN and Swin Transformer, namely CSwinDoubleU-Net, which takes full advantage of complementary information of both convolution and self-attention is proposed for polyp segmentation. • The first U-shaped codecs network utilizes the multi-convolution layers to extract the local feature information and the coordinate attention module in each skip connection. • The second U-shaped codec employs the Swin transformer layer to extract the global feature information. • CSFFM between two U-shaped codecs can enhance multi-level feature representation by fusing the low-level and global-level features in an interactive embedding manner, which further improves the prediction capability of the model by retrieving the information lost during feature extraction. The early detection of polyps from colonoscopy can reduce the risk of colorectal cancer development and give timely treatment. Accurate polyp segmentation during colonoscopy examinations can help clinicians locate the polyps which is of great significance in the clinical prevention of colorectal cancer. However, due to large variations in terms of size, color, texture, and morphology of polyps, the similarity between the polyp lesions and their background, the variations of illumination, motion blur, low-contrast areas, intestinal contents during image acquisitions, precisely polyp segmentation is still an open issue. To overcome the challenges above, a new double U-shaped image segmentation network combining convolution structure and Shifted Windows (Swin) Transformer, namely CSwinDoubleU-Net, is proposed in this paper. The developed CSwinDoubleU-Net consists of two U-shaped encode-decode structures associated with a pure CNN-based structure and a pure Transformer-based one, and a feature fusion module that interacts with the two U-shaped parts. The first U-shaped codecs network utilizes the multi-convolution layers to extract the local feature information and the coordinate attention module in each skip connection to effectively reduce the loss of spatial information and obtain the accurate position information of the encoded features. Next, the second U-shaped codecs network uses the Swin Transformer layers with the sliding window to extract global feature information further. At last, a convolutional feature and self-attention feature fusion module (CSFFM) is designed to deeply fuse the local convolutional features extracted from the first U-shape structure and the global self-Attention features extracted from the second U-shape structure. The obtained multi-category and multi-dimensional fused feature information can help to recover the boundary features of polyps. Extensive experiments are conducted to validate the proposed CSwinDoubleU-Net on five publicly available datasets, including CVC-ClinicDB, Kvasir, CVC-ColonDB, CVC-T, and ETIS-Larib. The results show that the proposed model can outperform some state-of-the-art methods and achieve high segmentation performance for polyp images. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel neural network architecture called CTCNet. First, we adopt a multi-scale convolutional neural network (MSCNN) to extract low and high-frequency features, adaptive channel feature recalibration (ACFR) to enhance the model's sensitivity to important channel features in the feature maps and reduce dependence on irrelevant or redundant features, a multi-scale dilated convolutional block (MSDCB) to capture characteristics of different types among feature channels. Second, we use Transformer to extract global temporal context features. Third, we employ capsule network to capture spatial location relationships among EEG features and refine these features. Besides, the capsule network module is used as our model's classifier to classify the final results. It is worth noting that our model better solves the problem that previous researches failed to take into account the simultaneous extraction of local features and global temporal context characteristics of EEG signals, and ignored the spatial location relationships between these features. Eventually, we assess our model on three datasets and it achieves better or comparable performance than most state-of-the-art methods. • The proposed method extracts the local and global features of EEG signals. • We design a multi-scale CNN to extract different frequency features of EEG signals. • We use Transformer to capture global temporal context information of EEG signals. • We use capsule network to extract spatial relationships between EEG features. • The accuracy on Sleep-EDF-20 is 86.2%, on Sleep-EDF-78 is 82.5% and on SHHS is 85.7%. [ABSTRACT FROM AUTHOR]

Machine learning models are widely used for identifying malicious software. However, existing models suffer from issues such as imprecise polysemous representations and a lack of contextual semantic representations, leading to the failure to recognize certain types of malicious software. In this paper, we propose a semantic-based intelligent malware detection model called SeMalBERT for identifying malicious software in Windows systems. Specifically, the model utilizes the API function sequences of malicious software as the learned features. Firstly, BERT is applied to accomplish word representation tasks and extract semantic information from the sequences. Secondly, a hybrid discriminator based on Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) is used to explore the chaining relationships between functions. Lastly, an external attention mechanism is added after the LSTM to enable the model to better focus on key information within the text. Experimental results demonstrate that SeMalBERT outperforms existing malware detection techniques in terms of accuracy, F1 score, and loss function value on a general dataset. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (CNNs) depend on feedforward and feedback ways to obtain good performance in image denoising. However, how to obtain effective structural information via CNNs to efficiently represent given noisy images is key for complex scenes. In this paper, we propose a cross Transformer denoising CNN (CTNet) with a serial block (SB), a parallel block (PB), and a residual block (RB) to obtain clean images for complex scenes. A SB uses an enhanced residual architecture to deeply search structural information for image denoising. To avoid loss of key information, PB uses three heterogeneous networks to implement multiple interactions of multi-level features to broadly search for extra information for improving the adaptability of an obtained denoiser for complex scenes. Also, to improve denoising performance, Transformer mechanisms are embedded into the SB and PB to extract complementary salient features for effectively removing noise in terms of pixel relations. Finally, a RB is applied to acquire clean images. Experiments illustrate that our CTNet is superior to some popular denoising methods in terms of real and synthetic image denoising. It is suitable to mobile digital devices, i.e., phones. Codes can be obtained at https://github.com/hellloxiaotian/CTNet. • Depth and breadth can search more structural information to improve image denoising performance. • Achieving multiple interactions can improve adaptability of denoiser for complex scenes. • Cross Transformer technique extracts more salient information, according to pixel relations for image denoising. [ABSTRACT FROM AUTHOR]

• The proposed TB-MFCC multifuse feature is comprised of data augmentation and feature extraction. • First, Pooled Multifuse Feature Augmentation uses the noise injection, stretching, shifting and pitching separately to increase the number of instances in the dataset and reduces overfitting. • Second, Triangular Bluestein Mel Frequency Cepstral Coefficients (TB-MFCC) are proposed and fused with Zero Crossing Rate (ZCR), Mel-frequency cepstral coefficients (MFCC) to extract the exact feature which increases the accuracy and reduces the Mean Squared Error (MSE). • Finally, the proposed MCNN with A-BiLSTM improves the nonlinear relationship among the features. • Thus the proposed augmentation, segmentation and classification is used to increase the accurate prediction of vehicle with reduced loss, less chance of overfitting and underfitting and avoids vanishing gradient problems. Vehicles equipped for emergencies like ambulances, fire engines, and police cruisers play a vital role in society by responding quickly to emergencies and helping to prevent loss of life and maintain order. Vehicle sound identification and classification are very important in the cities to identify emergency vehicles easily and to clear the traffic effectively. Convolutional Neural Network plays an important role in the accurate prediction of vehicles during an emergency. The main motive of this paper is to develop a suitable model and algorithms for data augmentation, feature extraction, and classification. The proposed TB-MFCC multifuse feature is comprised of data augmentation and feature extraction. First, in the proposed signal augmentation, each audio signal uses noise injection, stretching, shifting, and pitching separately and this process increases the number of instances in the dataset. The proposed augmentation reduces the overfitting problem in the network. Second, Triangular Bluestein Mel Frequency Cepstral Coefficients (TB-MFCC) are proposed and fused with Zero Crossing Rate (ZCR), Mel-frequency cepstral coefficients (MFCC), Root Mean Square (RMS), Chroma, and Tempogram to extract the exact feature which increases the accuracy and reduces the Mean Squared Error (MSE) of the model during classification. Finally, the proposed Multi-stacked Convolutional Neural Network (MCNN) with Attention-based Bidirectional Long Short Term Memory (A-BiLSTM) improves the nonlinear relationship among the features. The proposed Pooled Multifuse Feature Augmentation (PMFA) with MCNN & A-BiLSTM increases the accuracy (98.66 %), reduces the False Positive Rate (FPR) by 1.01 %, and loss (0 %). Thus the model predicts the sound without overfitting, underfitting, and vanishing gradient problems. [ABSTRACT FROM AUTHOR]

Brain tumors are one of the most threatening causes of death worldwide. Neuroradiologists widely use Magnetic Resonance Imaging as a golden standard non-invasive imaging method for diagnosing Glioma grades. Brain tumor diagnosis based on radiology images is tedious and highly subjective to intra and inter-observer variability; it adversely affects therapeutic planning. Better medical treatment can be provided relating to similar past cases if relevant images are retrieved from an extensive medical image database. Content-Based Image Retrieval systems are potent image tools for dealing with such massive datasets. In a CBIR system, accurate classification and retrieval of similar pathological images can be effectively automated by utilising Convolutional Neural Network-based feature extraction methods. This paper presents a Content-Based Medical Image Retrieval pipeline on a medical domain using the CNN model for feature extraction and the clustering method used to index the feature map database. The proposed system applies a Multi-level Gain-based feature selection to reduce the dimensionality of the feature vectors obtained from the pre-trained CNN models. The experiment follows five-fold cross-validation on BraTS 2018 and 2020 datasets. The proposed system achieves state-of-the-art MRI brain image retrieval performance with a mean Average Precision and Precision@10 of 98.15% and 97.62%, respectively. The results prove the proposed method's effectiveness and validate its clinical application. • Inventing a proper CBMIR pipeline for medical domain with state-of-the-art efficiency. • Figured out relevant components of the pipeline using theoretical and experimental trials. • Proposed a novel feature space reduction strategy using Multi-level Gain-based feature selection. • Feature vector indexing for fast and efficient MR image retrieval from large database. [ABSTRACT FROM AUTHOR]

In human–computer dialogue systems, intent recognition is crucial for determining the intentions or purposes of users during interactions with the system, enabling the system to provide appropriate responses or actions. This paper proposes an intent recognition model that integrates sequential information and sentence structural features. Specifically, the approach utilizes a CNN to capture local salient features in the text, followed by a BILSTM to extract sequential information within the local context. The sequential information is then fed into a multi-head attention mechanism to focus on more relevant sequential details. Additionally, the original data is processed by BERT to extract sentence structural features. Finally, the sequential information features and sentence structural features are concatenated and fused to achieve enhanced intent recognition performance. This approach effectively leverages contextual and semantic information within the text, leading to improved accuracy in intent recognition. Experimental results demonstrate the effectiveness of the proposed method in intent recognition and its high relevance for practical applications. [ABSTRACT FROM AUTHOR]

[Display omitted] • Proposing a novel approach to predict energy consumption in mosque buildings during the operation stage. • Simulating various scenarios involving different operational variables of the mosque and the utilized systems. • Developing a Convolution Neural Network (CNN) deep learning model to predict the mosques' annual energy consumption. • Comparing the developed deep learning model against the Support Vector Regression (SVR) model. The energy consumption resulting from the construction sector is increasing rapidly. Different building types have different energy consumption schemes, especially during their operational stage. Among these types of buildings, mosques are considered one of the intermittent building types that suffer from huge energy wastage. That's why there is a need to monitor their operational performance through retrofitting or changing their operational scheme to enhance their operational performance and examine the effect of different alterations on the annual energy consumption. However, using traditional methods and simulation models to examine the impact of different alterations on the annual energy consumption of the mosque is considered difficult and time-consuming. Using machine learning and deep learning models, energy prediction models facilitated energy consumption predictions for improving building energy utilization. Therefore, this paper proposes an efficient approach for predicting mosques' energy consumption using a deep learning approach. Convolution Neural Network (CNN) deep learning model is developed to predict the annual energy consumption of mosque buildings based on various operational Scenarios. 3D Laser scanning and a detailed energy analysis model with various simulation results are used to generate the mosque's simulated energy consumption dataset. Different operational variables, such as the operational schedule, division of the mosque into zones, utilization of dimmers, presence of cooling system and the setting point, and the power consumption of the lighting system and appliances, have been utilized to estimate the annual energy consumption for the mosque to formulate the database. Using the mosque's simulated energy consumption dataset, the CNN model was trained and tested to obtain the best model configuration with regard to the defined performance metrics. Besides, the performance of the developed model has been compared with the Support Vector Regression (SVR) model. The developed model has achieved results of 4.5% and 0.98 for the MAPE and R2, respectively. The results revealed the developed CNN model's effectiveness for predicting energy consumption for mosque buildings. This model will help compare different operational strategies for mosque buildings and determine energy consumption and energy-efficient options based on different operational characteristics. [ABSTRACT FROM AUTHOR]

• It is proved that the elastic wave field excited by the leakage source in the wall is related to the leakage position. • Not limited to a particular moment (like AE), this method can locate the leak source at any time after the leak occurs. • This method simplifies the original multi-sensor array into one and reduces the complexity of real-time leak location system. To satisfy the requirement of real-time monitoring and locating of the leakage in large pressure vessels, a method of leakage source location based on improved convolutional neural network (CNN) is proposed in this paper. In this method, the polar coordinate model of elastic wave propagation excited by continues leakage is established, and the improved CNN is constructed according to its spatial field distribution, and the training set of the network is obtained by finite element simulation. The experimental results show that this method can be applied to locate the leak at any time after it occurs, and only a single sensor array is used. In the experiment of 1 m2 stiffened test plate, the absolute distance error was less than 0.1 m and the absolute angle error was less than 0.1 rad in 90.2 % of all samples. [ABSTRACT FROM AUTHOR]

In this paper, a Raman spectral detection and identification method combined with a portable Raman device is proposed to realize the real-time detection of trace substances. By employing a composite Surface-Enhanced Raman Scattering (SERS) substrate coupled with surface plasmon polaritons (SPP) and localized surface plasmons (LSP), the Raman detection performance for trace can be improved significantly. The one-dimensional dilated convolutional neural network (1D CNN) model proposed in this paper combined with a spatial attention mechanism can accurately identify Raman spectra. Together, a combination of the two provides great advantages, especially for the identification of weak signals with indistinguishable characteristic peaks. At the same time, for the public Raman dataset Rruff, the model has higher accuracy and generalization ability than traditional Raman spectral recognition algorithms. The evidence presented in this paper reveals that the LSP–SPP​ coupled SERS substrate combined with a 1D CNN-based spectral identification algorithm can pave the way for the development of an accurate and portable Raman detection of trace. • A convenient LSP–SPP coupled SERS substrate is proposed. • A SERS spectrum recognition model based on a one-dimensional (1D) dilated CNN combined with spatial attention mechanisms is designed. [ABSTRACT FROM AUTHOR]

This paper introduces the Ongoing Event Detection (OED) task, which is a specific Event Detection task where the goal is to detect ongoing event mentions only, as opposed to historical, future, hypothetical, or other forms or events that are neither fresh nor current. Any application that needs to extract structured information about ongoing events from unstructured texts can take advantage of an OED system. The main contribution of this paper are the following: (1) it introduces the OED task along with a dataset manually labeled for the task; (2) it presents the design and implementation of an RNN model for the task that uses BERT embeddings to define contextual word and contextual sentence embeddings as attributes, which to the best of our knowledge were never used before for detecting ongoing events in news; (3) it presents an extensive empirical evaluation that includes (i) the exploration of different architectures and hyperparameters, (ii) an ablation test to study the impact of each attribute, and (iii) a comparison with a replication of a state-of-the-art model. The results offer several insights into the importance of contextual embeddings and indicate that the proposed approach is effective in the OED task, outperforming the baseline models. • The Ongoing Event Detection (OED) task is defined. • A dataset for the task is made available, along with extensive documentation. • Baselines for the task using classical and state-of-the-art techniques are reported. • A proposed model that uses contextual word and sentence embeddings is presented. • An extensive empirical evaluation is reported with several variations of the models. [ABSTRACT FROM AUTHOR]

With the rise in popularity of social media platforms and online forums, they have become a global source of news. Fake News (FNs) spreading through numerous institutions and sectors jeopardizes their reputations, causing users to abandon these platforms. Therefore, there is a huge pool of research in the area of Artificial Intelligence (AI) techniques that are used to detect FNs. Previously, great focus was given to online review classification and free internet posts based on social networks. This research proposes a Deep Learning-based FNs Detection method. This paper proposes a Deep Learning (DL)-based method for detecting FNs. The proposed system consists of three phases; text encoding, feature extraction, and classification. The text encoding process is carried out on the input news words using GLOVE for word representation. The encoded words are then embedded into a specific word length in order to be enrolled in the proposed DL models. The proposed DL models comprise both automatic feature extraction and classification tasks. Furthermore, this study proposes four different DL models, including Convolutional Neural Networks (CNNs) and Concatenated CNNs (C-CNNs), long short-term memory (LSTM), and Gated Recurrent Units, to find an optimal model prior to the section of FNs that outperforms previous works.The proposed DL models are carried out on FNs and FNC datasets which are provided by kaggle, and the suggested C-CNNs algorithm obtained an accuracy of 99.6% and trained faster than others. Multiple evaluation metrics such as precision, recall, F1, and accuracy have been utilized to evaluate the outcome of the proposed models. The experimental results demonstrated overall improvements in the subject of FND when compared with the current models and validated the potential of the proposed methodology for the detection of FNs on Social Media (SM). This study will help researchers to broaden the knowledge of applications of CNNs based on DL methods for FND. • This paper proposes a solution for fake news detection based on deep learning. • The proposed deep learning approach comprises both sequential and recurrent modalities. • The proposed sequential models includes CNNs and concatenated CNNs, while the recurrent includes LSTM and GRU models. • Each model is compared with the other proposed models as well as the works in the literature. [ABSTRACT FROM AUTHOR]

• An end to end framework for clothes image captioning is developed based on attribute detection and visual attention. • The attribute detection module is developed to provide the encoded feature of clothes with more attribute information. • An attention gate is adopted to better characterize the similarity between image portion and captioning description. • Extensive simulations based on real-world data are conducted to verify the effectiveness of the proposed method. Fashion is a multi-billion-dollar industry, which is directly related to social, cultural, and economic implications in the real world. While computer vision has demonstrated remarkable success in the applications of the fashion domain, natural language processing technology has become contributed in the area, so that it can build the connection between clothes image and human semantic understandings. An element work for combing images and language understanding is how to generate a natural language sentence that accurately summarizes the contents of a clothes image. In this paper, we develop a joint attribute detection and visual attention framework for clothes image captioning. Specifically, in order to involve more attributes of clothes to learn, we first utilize a pre-trained Convolutional Neural Network (CNN) to learn the feature that can characterize more information about clothing attribute. Based on such learned feature, we then adopt an encoder/decoder framework, where we first encoder the feature of clothes and then and input it to a language Long Short-Term Memory(LSTM) model for decoding the clothes descriptions. The method greatly enhances the performance of clothes image captioning and reduces the misleading attention. Extensive simulations based on real-world data verify the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

• A deep learning method for inspection of exterior substance is proposed. • Dense connection and spatial pyramid pooling are used to extract features. • Data augmentation and transfer learning was used to build inspection model. • The proposed model significant outperformed other deep learning method. When a high-speed train is running, exterior substances, such as rail-side plastic bags, are easily rolled into the bogies, cables and equipment gaps, which can easily cause smoke, odor and even equipment short-circuits and fires. Current inspection methods have many disadvantages. To overcome these defects, this paper presents a fast intelligent inspection model based on the YOLO V3 deep learning method for exterior substance inspection. We utilized the densely convolutional network as the feature extractor to enhances the feature propagation and ensure maximum information flow in the YOLO V3 method. Spatial pyramid pooling networks was used in the YOLO V3 method to reduce context information loss. The transfer learning and data augmentation technologies were applied to make the present method easier to train. Compared with other method, the proposed method shows satisfactory performance in terms of precision and recall, which is the stare-of-art exterior substance inspection method. [ABSTRACT FROM AUTHOR]

• Rectification of problems due to sudden illumination changes and proper alignment of feature set for an effective FER. • Isolation of foreground from the background of an image to facilitate the facial key points extraction. • Elimination of unwanted noise and smoothen by using AMF filtering technique. • MATP-based texture pattern and DESURF-LOP-based facial key points to the FER outperform the traditional methods. • PPCSO-based feature selection and LSTM-CNN based classification recognize the expressions accurately. Facial Expression Recognition (FER) plays the vital role in the Human Computer Interface (HCI) applications. The illumination and pose variations affect the FER adversely. The projection of complex 3D actions on the image plane and the inaccurate alignment are the major issues in the FER process. This paper presents the novel Multi-Angle Optimal Pattern-based Deep Learning (MAOP-DL) method to rectify the problem from sudden illumination changes, find the proper alignment of a feature set by using multi-angle-based optimal configurations. The proposed method includes the five major processes as Extended Boundary Background Subtraction (EBBS), Multi-Angle Texture Pattern+STM, Densely Extracted SURF+Local Occupancy Pattern (LOP), Priority Particle Cuckoo Search Optimization (PPCSO) and Long Short-Term Memory -Convolutional Neural Network (LSTM-CNN). Initially, the EBBS algorithm subtracts the background and isolates the foreground from the images which overcome the illumination and pose variation. Then, the MATP-STM extracts the texture patterns and DESURF-LOP extracts the relevant key features of the facial points. The PPCSO algorithm selects the relevant features from the MATP-STM feature set to speed up the classification. The employment of LSTM-CNN predicts the required label for the facial expressions.The major key findings of the proposed work are clear image analysis, effective handling of pose/illumination variations and the facial alignment. The proposed MAOP-DL validates its effectiveness on two standard databases such as CK+ and MMI regarding various metrics and confirm their assurance of wide applicability in recent applications. [ABSTRACT FROM AUTHOR]

• An effective method for detection of multi-class brain abnormalities is proposed. • The proposed deep CNN model demands less learnable parameters. • The CNN helps to learn prominent features automatically from the brain MRIs. • The model achieves greater classification accuracy over benchmark datasets. Automated detection of multi-class brain abnormalities through magnetic resonance imaging (MRI) has received much attention due to its clinical significance and therefore has become an active area of research over the years. The earlier automated schemes often followed traditional machine learning paradigms, in which the proper choice of features and classifiers has remained a major concern. Therefore, deep learning algorithms have been profoundly applied in various medical imaging applications. In this paper, a deep convolutional neural network (CNN) based automated approach is designed for the diagnosis of multi-class brain abnormalities. The proposed CNN model comprises five layers with learnable parameters: four convolutional layers and one fully-connected layer. The objective of designing such a custom deep network is to achieve greater classification performance with reduced number of parameters. The proposed model is evaluated on two benchmark multi-class brain MRI datasets namely, MD-1 and MD-2. The model achieved a classification accuracy of 100.00% and 97.50% on MD-1 and MD-2 datasets respectively. Moreover, four pre-trained CNN models based on the transfer learning approach have been tested over the same datasets. The comparative analysis with existing schemes indicates the superiority of the proposed method. [ABSTRACT FROM AUTHOR]

Corona virus disease-2019 (COVID-19) is a pandemic caused by novel coronavirus. COVID-19 is spreading rapidly throughout the world. The gold standard for diagnosing COVID-19 is reverse transcription-polymerase chain reaction (RT-PCR) test. However, the facility for RT-PCR test is limited, which causes early diagnosis of the disease difficult. Easily available modalities like X-ray can be used to detect specific symptoms associated with COVID-19. Pre-trained convolutional neural networks are widely used for computer-aided detection of diseases from smaller datasets. This paper investigates the effectiveness of multi-CNN, a combination of several pre-trained CNNs, for the automated detection of COVID-19 from X-ray images. The method uses a combination of features extracted from multi-CNN with correlation based feature selection (CFS) technique and Bayesnet classifier for the prediction of COVID-19. The method was tested using two public datasets and achieved promising results on both the datasets. In the first dataset consisting of 453 COVID-19 images and 497 non-COVID images, the method achieved an AUC of 0.963 and an accuracy of 91.16%. In the second dataset consisting of 71 COVID-19 images and 7 non-COVID images, the method achieved an AUC of 0.911 and an accuracy of 97.44%. The experiments performed in this study proved the effectiveness of pre-trained multi-CNN over single CNN in the detection of COVID-19. [ABSTRACT FROM AUTHOR]

This paper proposes a simple and effective model applied for image-based malware classification using machine learning in which malware images (converted from malware binary files) are directly fed into the classifiers, i.e. k nearest neighbour (k -NN), support vector machine (SVM) and convolution neural networks (CNN). The proposed model does not use the normalized fixed-size square images (e.g. 64 × 64 pixels) or features extracted by image descriptor (e.g. GIST) for training classifiers as existing models do in the literature. Instead, the input images are normalized and horizontally sized down (the width of the image) to a lower dimension of 32 × 64, 16 × 64 or even 8 × 64 than square ones (e.g. 64 × 64 pixels) to reduce the complexity and training time of the model. It is based on the fact that the texture of the malware image is mainly vertically distributed as analysed in this paper. This finding is significant for training those devices which have limited computational resources such as IoT devices. The experiment was conducted on the Malimg, Malheur datasets which contains 9339 (25 malware families) and 3133 variant samples (24 malware families) using k -NN, SVM and CNN classifiers. The achieved results show that it is possible to reduce the dimension of the input images (i.e. 32 × 64, 16 × 64 or even 8 × 64) while still retaining the accuracy of classification as the same as the accuracy obtained by classifier feeding by the fixed-size square image (i.e. 64 × 64 pixels). As a result, training time of the propose model reduces by a half, a quarter, and one-eighth compared to training time taken by the same machine learning-based classifier (i.e. k -NN, SVM and CNN) feeding by fixed-sized square images, i.e. 64 × 64, respectively. [ABSTRACT FROM AUTHOR]

Music is often considered as the language of emotions. The way it stimulates the emotional appraisal across people from different communities, culture and demographics has long been known and hence categorizing on the basis of emotions is indeed an intriguing basic research area. Indian Classical Music (ICM) is famous for its ambiguous nature, i.e. its ability to evoke a number of mixed emotions through only a single musical narration, and hence classifying evoked emotions from ICM becomes a more challenging task. With the rapid advancements in the field of Deep Learning, this Music Emotion Recognition (MER) task is becoming more and more relevant and robust, hence can be applied to one of the most challenging test case i.e. classifying emotions elicited from ICM. In this paper we present a new dataset called JUMusEmoDB which presently has 1600 audio clips (approximately 30 s each) where 400 clips each correspond to happy, sad, calm and anxiety emotional scales. The initial annotations and emotional classification of the database was done based on an emotional rating test (5-point Likert scale) performed by 100 participants. The clips have been taken from different conventional ' raga' renditions played in two Indian stringed instruments – sitar and sarod by eminent maestros of ICM and digitized in 44.1 kHz frequency. The ragas , which are unique to ICM, are described as musical structures capable of inducing different moods or emotions. For supervised classification purposes, we have used Convolutional Neural Network (CNN) based architectures (resnet50, mobilenet v2.0, squeezenet v1.0 and a proposed ODE-Net) on corresponding music spectrograms of the 6400 sub-clips (where every clip was segmented into 4 sub-clips) which contain both time as well as frequency domain information. Along with emotion classification, instrument classification based response was also attempted on the same dataset using the CNN based architectures. In this context, a nonlinear technique, Multifractal Detrended Fluctuation Analysis (MFDFA) was also applied on the musical clips to classify them on the basis of complexity values extracted from the method. The initial classification accuracy obtained from the applied methods are quite inspiring and have been corroborated with ANOVA results to determine the statistical significance. This type of CNN based classification algorithm using a rich corpus of Indian Classical Music is unique even in the global perspective and can be replicated in other modalities of music also. The link to this newly developed dataset has been provided in the dataset description section of the paper. This dataset is still under development and we plan to include more data containing other emotional as well as instrumental entities into consideration. • A novel database of 1600 emotional clips taken from Indian Classical music (ICM). • Application of nonlinear and CNN based methods to classify emotions and instruments from ICM. • Proposal of a new ODE-Net based architecture which provides the best classification rate. • Multifractal technique reflects the inherent ambiguity in emotions of ICM. [ABSTRACT FROM AUTHOR]

In recent years, convolutional neural networks (CNNs) have achieved success in the field of computer vision. However, their large storage requirements and high computational consumption limit their scope of application, especially, in mobile devices. Network pruning has been proven to be an effective approach for relieving such burdens. However, the pruning ratio and accuracy of the existing methods need to be improved. In this paper, a scheme of network channel pruning, based on sparse learning and the genetic algorithm, is proposed to achieve a better balance between the pruning ratio and accuracy. A gradient-limited L 1/2 regularization is introduced into the model training, to achieve channel-level sparsity. In addition, a dynamically adjustable evaluation factor is proposed to evaluate the importance of channels and the genetic algorithm is utilized to acquire a suitable subnetwork from the sparse model. Our method is evaluated against several state-of-the-art CNNs on three different classification datasets, and is found to achieve, on an average, 63.8%/50.8%/58.6% reduction in parameters and 49.1%/42.9%/58.7% reduction in computational complexity, with negligible loss of accuracy. Our method also achieves 42.84% reduction in parameters and 30.06% reduction in computational complexity of the detection network. [ABSTRACT FROM AUTHOR]

Fusion of panchromatic (PAN) and multispectral (MS) images, called pansharpening, is one of the main challenging problems in remote sensing. Convolutional neural network (CNN) based pansharpening has been introduced in some works recently. The represented frameworks often use a multi-layer CNN for fusion of MS and PAN images. Limited number of training samples and the high number of network weights to be determined may cause overfitting problem. To deal with this difficulty, a simple structure comprised from two single layer convolutional networks is proposed in this paper. The shortage of deleted layers is compensated by applying 3D Gabor filters and shearlet transform in addition to nonlinear kernel based principal component analysis. The proposed framework not only has a simple structure, but also is superior to state-of-the-art CNN based pansharpening methods in terms of various quality assessment measures. [ABSTRACT FROM AUTHOR]

Sign Language Recognition (SLR) is a relatively popular research area yet contrary to its popularity, the implementation of SLR in daily basis is rare; this is due to the complexity and various resources required. In this literature review, the authors have analyzed various techniques that can be used to implement an automated sign-language translator through the analysis of the methodologies and models used to make a working model of any sign-language translator from various sources. The purpose of this study is to explore various possible ways to implement Artificial Intelligence technology to improve the automated American Sign Language translator that is applicable. The authors have identified 22 different research papers within the period of the years 2015 - 2020. The analysis showed that every research studies picked have achieved respectable results, however, they are not perfect, since each research demonstrates its own unique strengths and weaknesses. There are some methods that might be suitable for our need to create an applicable Sign Language Translator, that is by using standard video camera for obtaining data, and either Convolutional Neural Network or Support Vector Machine can be used for the classification. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNN) have shown its excellent performance in computer vision fields. Recently, they are successfully applied to image restoration. This paper proposes a joint blur kernel estimation and CNN method for blind image restoration. The blur kernel estimation is based on both blur support parameter estimation and blur type identification. An automatic feature line detection algorithm is presented for blur support parameter estimation and a dictionary learning algorithm is presented for the blur type identification. Once the blur kernel estimate is obtained, we use an effective CNN for iterative non-blind deconvolution, which is able to automatically learn image priors. Compared with current blind image restoration methods, the proposed joint method can obtain restored images under three types of unknown blur kernels. The experimental result shows that the proposed blur kernel estimation algorithm can provide high accuracy results. Furthermore, the proposed joint blur kernel estimation and CNN algorithm is superior to conventional blind image restoration algorithms in terms of restoration quality and computation time. [ABSTRACT FROM AUTHOR]

• The SolarNet generates accurate intra-hour solar forecasts from a single sky image. • The SolarNet outperforms machine learning models under various weather conditions. • The SolarNet has 8.85% forecasting nRMSE and a 25.14% forecasting skill score. The exponential growth of solar energy poses challenges to power systems, mostly due to its uncertain and variable characteristics. Hence, solar forecasting, such as very short-term solar forecasting (VSTSF), has been widely adopted to assist power system operations. The VSTSF takes inputs from various sources, among which sky image-based VSTSF is not yet well-studied compared to its counterparts. In this paper, a deep convolutional neural network (CNN) model, called the SolarNet, is developed to forecast the operational 1-h-ahead global horizontal irradiance (GHI) by only using sky images without numerical measurements and extra feature engineering. The SolarNet has a set of models that generate fixed-step GHI in parallel. Each model is composed of 20 convolutional, max-pooling, and fully-connected layers, which learns latent patterns between sky images and GHI in an end-to-end manner. Numerical results based on six years data show that the developed SolarNet outperforms the benchmarking persistence of cloudiness model and machine learning models with an 8.85% normalized root mean square error and a 25.14% forecasting skill score. The SolarNet shows superiority under various weather conditions. [ABSTRACT FROM AUTHOR]

• A novel frame extraction mechanism that works on videos on the fly. • Extract frames from input video containing human actions only with no delay. • A typical CNN heuristic for efficient human action recognition. • Performance in various measures found to be outperforming among competent schemes. Recognition of human actions with optimum accuracy and with less computational overhead has always been a challenging task. A suitable framework towards this problem can provide a robust solution for numerous domains of expertise that demands automatic surveillance. In this paper, a two-fold framework has been proposed for the frame extraction and recognition of human action therein from a video input. A Fuzzy inferencing technique has been suitably used for the purpose of video frame extraction in a smarter way. Frames are extracted from a video only when an event of action is initiated and about to commit. By this the additional computational load of continuous frame extraction is eased efficiently. Subsequent to this process, we also introduce a typical convolutional neural network (CNN) that is used for the purpose of human action recognition. This typical CNN has been designed so as to suit the nature of input that is supplied to it. Experimental evaluation has been performed on the standard HMDB51 database. A total of eight distinct and prominent actions which are mostly common in daily routine of human life are considered for the purpose. Performance comparison of the proposed scheme with other state of the art schemes is also performed with three different measures. Evaluation results for the proposed scheme indicates overall rate of accuracies of 96.5% and 98% for frame extraction and action recognition respectively. The proposed scheme outperformed the others in most of the measuring aspects. [ABSTRACT FROM AUTHOR]

• Promising pruning methods utilize empirical classification loss to remove parameters of CNNs. • Empirical classification loss, however, might be vulnerable to outlier samples. • We propose a novel pruning criterion to be robust against the outlier samples. • Our method also provides stable metric for parameters. • Experiment results demonstrate favorable performance of our method in comparison with the other pruning methods. Deep convolutional neural network (CNN) with considerable number of parameters is one of the promising methods for image recognition. There, however, is generally difficult in applying deep CNNs to resource constrained devices due to the heavy computational burden. For reducing computational cost of CNNs while retaining the classification performance, it is effective to apply pruning methods that remove from CNNs redundant parameters less contributing to classification. The contribution of parameters can be estimated by the empirical classification loss computed over training samples to which ground-truth labels are assigned. The empirical classification loss, however, might be vulnerable to the outlier samples and/or the hard ones that are difficult to classify, and thus the pruning would accordingly be degraded. In this paper, we propose a pruning method based on a novel criterion to measure the redundancy of the parameters in CNNs through empirical classification loss. We start with the Taylor expansion of the loss function and then derive the mathematical formulation of the pruning criterion so as to be robust against some sort of outlier samples. The proposed pruning criterion can also provide stable metric for parameters and evaluate layers of various depth fairly without biases toward shallower or deeper layers. In addition, we present an effective method to normalize the criterion scores for further improving performance. In the experiments on image classification, our method exhibits favorable performance compared with the other methods. [ABSTRACT FROM AUTHOR]

A brain tumor is an abnormal growth of cells inside the skull. Malignant brain tumors are among the most dreadful types of cancer with direct consequences such as cognitive decline and poor quality of life. Analyzing magnetic resonance imaging (MRI) is a popular technique for brain tumor detection. In this paper, we use these images to train our new hybrid paradigm which consists of a neural autoregressive distribution estimation (NADE) and a convolutional neural network (CNN). We subsequently test this model with 3064 T1-weighted contrast-enhanced images with three types of brain tumors. The results demonstrate that the hybrid CNN-NADE has a high classification performance as regards the availability of medical images are limited. [ABSTRACT FROM AUTHOR]

We present a practical approach to improve the precision of face alignment for a single image. Recently, face alignment is deemed as a regression problem, and convolutional neural networks (CNNs) or recurrent neural networks (RNNs) are utilized to predict the coordinates of facial landmarks. However, most existing methods only adopt Euclidean loss as the optimization target for each landmark, and neglect the correlations between them, which we think may be inappropriate. To address this issue, in this paper, we introduce a novel Relative Coordinates Constraint (RCC) loss function for face alignment, which considers the relative coordinates between any pairs of landmarks as a new supervision signal. More importantly, we prove that the proposed RCC loss function is trainable and can be easily incorporated in existing CNNs optimization procedure. With the joint supervision of Euclidean loss and RCC loss, we train a robust and light CNNs framework for face alignment. Extensive experimental results on several datasets show that the precision of face alignment improved significantly by the proposed RCC loss and quantitative results are comparable to state-of-the-art methods (mean error 5.39 on 300-W and 6.99 on AFLW). In addition, the proposed framework is also an efficient solution (300 FPS on CPU). We share the implementation code of our proposed methods at https://github.com/nianfudong/RCC-loss. [ABSTRACT FROM AUTHOR]

Image caption approaches that use the global Convolutional Neural Network (CNN) features are not able to represent and describe all the important elements in complex scenes. In this paper, we propose to enrich the semantic representations of images and update the language model by proposing semantic element embedding. For the semantic element discovery, an object detection module is used to predict regions of the image, and a captioning model, Long Short-Term Memory (LSTM), is employed to generate local descriptions for these regions. The predicted descriptions and categories are used to generate the semantic feature, which not only contains detailed information but also shares a word space with descriptions, and thus bridges the modality gap between visual images and semantic captions. We further integrate the CNN feature with the semantic feature into the proposed Element Embedding LSTM (EE-LSTM) model to predict a language description. Experiments on MS COCO datasets demonstrate that the proposed approach outperforms conventional caption methods and is flexible to combine with baseline models to achieve superior performance. [ABSTRACT FROM AUTHOR]

The Coronavirus epidemic first appeared in Wuhan-China on December, 31st, 2019. This has put the world's hospitals, clinics, testing laboratories and health administrations under pressure. As of April, 04th, 2020, the World Health Organization reported more than 167515 confirmed cases in more than 100 countries worldwide. The diagnosis of the epidemic will increase the burden on overburdened testing laboratories. Several screening methods have been proposed in parallel in order to facilitate and, above all, to make rapid diagnosis easier. At this level, X-Ray images seem to be a good accompanying solution for emerging countries to help rapid screening. The solution proposed in this paper consists on a collaborative and smart platform based on the Convolutional Neural Network for Classification and Detection namely VGG16. The platform ensures the fast download of the X-Ray image, with the entry of the patient's personal information followed by the launch of a 5 seconds test. The platform generates, as a result, a PDF file containing all patient information. [ABSTRACT FROM AUTHOR]

Object tracking is a problem about semi-supervised learning with insufficient data set. In the field of military navigation and security of public life, it is widely used to take the place of human beings. In this paper, we come up with a new algorithm based on Bayesian, CNN and PLK optical flow, which is called COB method, for object tracking problems. With the idea of track-by-detect, we cascade CNN after PLK optical flow and integrate them in a Bayesian method. Most importantly our method is proposed with an adaptive integrating method to reduce the influence of over-fitting. The integrator also introduces the competition mechanism between tracker and detector, so that the algorithm is able to update the classifier with online learning. Besides, the regularization of deep learning is used to solve the blind spots of classifier. The experimental results show that the algorithm is more robust than the previous work. [ABSTRACT FROM AUTHOR]

As a method to judge the benign or malignant polyp from endoscope images, some methods have been proposed using an ultra-high magnification endoscope. The ultra-high magnification endoscope enables the diagnosis at the cell level. However, it tends to spend many times for diagnosis and requires specific expensive devices. There are three types of images that are taken for diagnosis by the regular endoscope: white light, dye, and narrowband image (NBI) in general. This paper proposes a benign/malignant polyp classification method using these images taken by the regular endoscope. Each image features derived from endoscope images are extracted by adapting a pre-trained CNN to each domain. Finally, polyps are classified using extracted features. Experiments confirmed that the proposed method enabled the classification of benign or malignant colorectal polyps with over 90% accuracy. [ABSTRACT FROM AUTHOR]

In the time on structured surfaces (walls, roofs, bridges, streets, etc.) cracks appear and influence from the aesthetic point of view, but also from the point of view of their resistance and quality. Traditionally, crack detection is performed by human visual inspection, which is dangerous (when they need to climb buildings), subjective (depending on their experience in detecting the severity of a crack), and time-consuming (if we consider hundreds of buildings). Increasingly, artificial intelligence continues to evolve and we can use it to improve human performance and automate the process of crack detection. In order to improve this problem, we present an application that detects cracks in buildings that are difficult to access or would endanger human life. The architecture of our application is based on Convolutional Neural Network. In this paper, three different approaches are described and compared. [ABSTRACT FROM AUTHOR]