Search

Improving the classification accuracy of electrocardiogram (ECG) signals is of great significance for diagnosing heart abnormalities and arrhythmias and preventing cardiovascular diseases (CVDs). The traditional classification method depends on medical experience to select and extract features artificially, lacks the generalization ability to deal with big medical data. The vital feature extraction ability of neural networks has become a hot topic to solve this problem. Based on this, the capsule network is applied to ECG signal classification in this paper. Based on the original network architecture, long short-term memory (LSTM) network and 1D convolutional neural network (CNN) are added as a parallel feature extraction layer to extract the spatial and temporal features of the ECG signal. In addition, the enhanced routing algorithm is proposed, which uses the prior probability of sub-capsules as a weighting factor for routing algorithm classification to weaken the influence of noise capsules. The proposed model is superior to the existing state-of-the-art techniques when tested on the MIT-BIH arrhythmia database. [ABSTRACT FROM AUTHOR]

Convolutional Neural Networks (CNN) are playing a vital role nowadays in every aspect of computer vision applications. In this paper we have used the state of the art CNN in recognizing handwritten Tamil characters in offline mode. CNNs differ from traditional approach of Handwritten Tamil Character Recognition (HTCR) in extracting the features automatically. We have used an isolated handwritten Tamil character dataset developed by HP Labs India. We have developed a CNN model from scratch by training the model with the Tamil characters in offline mode and have achieved good recognition results on both the training and testing datasets. This work is an attempt to set a benchmark for offline HTCR using deep learning techniques. This work have produced a training accuracy of 95.16% which is far better compared to the traditional approaches. [ABSTRACT FROM AUTHOR]

[Display omitted] • Artificially determining the degree of fermentation of black tea was arbitrary. • Compare the experimental results and select the student model and teacher model. • Focal Loss was introduced to improve the discriminative performance of the model. • The best result was achieved with the MGD method at distillation loss rate of 0.8. Black tea is the second most common type of tea in China. Fermentation is one of the most critical processes in its production, and it affects the quality of the finished product, whether it is insufficient or excessive. At present, the determination of black tea fermentation degree completely relies on artificial experience. It leads to inconsistent quality of black tea. To solve this problem, we use machine vision technology to distinguish the degree of fermentation of black tea based on images, this paper proposes a lightweight convolutional neural network (CNN) combined with knowledge distillation to discriminate the degree of fermentation of black tea. After comparing 12 kinds of CNN models, taking into account the size of the model and the performance of discrimination, as well as the selection principle of teacher models, Shufflenet_v2_x1.0 is selected as the student model, and Efficientnet_v2 is selected as the teacher model. Then, CrossEntropy Loss is replaced by Focal Loss. Finally, for Distillation Loss ratios of 0.6, 0.7, 0.8, 0.9, Soft Target Knowledge Distillation (ST), Masked Generative Distillation (MGD), Similarity-Preserving Knowledge Distillation (SPKD), and Attention Transfer (AT) four knowledge distillation methods are tested for their performance in distilling knowledge from the Shufflenet_v2_x1.0 model. The results show that the model discrimination performance after distillation is the best when the Distillation Loss ratio is 0.8 and the MGD method is used. This setup effectively improves the discrimination performance without increasing the number of parameters and computation volume. The model's P, R and F1 values reach 0.9208, 0.9190 and 0.9192, respectively. It achieves precise discrimination of the fermentation degree of black tea. This meets the requirements of objective black tea fermentation judgment and provides technical support for the intelligent processing of black tea. [ABSTRACT FROM AUTHOR]

The existence of fundus diseases not only endangers people's vision, but also brings serious economic burden to the society. Fundus images are an objective and standard basis for the diagnosis of fundus diseases. With the continuous advancement of computer science, deep learning methods dominated by convolutional neural networks (CNN) have been widely used in fundus image classification. However, the current CNN-based fundus image classification research still has a lot of room for improvement: CNN cannot effectively avoid the interference of repeated background information and has limited ability to model the whole world. In response to the above findings, this paper proposes the CNN-Trans model. The CNN-Trans model is a parallel dual-branch network, which is the two branches of CNN-LSTM and Vision Transform (ViT). The CNN-LSTM branch uses Xception after transfer learning. As the original feature extractor, LSTM is responsible for dealing with the gradient disappearance problem in neural network iterations before the classification head, and then introduces a new type of lightweight attention mechanism between Xception and LSTM: Coordinate Attention, so as to emphasize the key information related to classification and suppress the less useful repeated background information; while the self-attention mechanism in the ViT branch is not limited by local interactions, it can establish long-distance dependence on the target and extract global features. Finally, the concatenation (Concat) operation is used to fuse the features of the two branches. The local features extracted by the CNN-LSTM branch and the global features extracted by the ViT branch form complementary advantages. After feature fusion, more comprehensive image feature information is sent to the to the classification layer. Finally, after a large number of experimental tests and comparisons, the results show that: the CNN-Trans model achieved an accuracy of 80.68% on the fundus image classification task, and the CNN-Trans model has a classification that is comparable to the state-of-the-art methods. performance.. [ABSTRACT FROM AUTHOR]

Determining the correct mushroom species with the necessary ecological characteristics is critical to continue mushroom production, which is essential in gastronomy. The mushroom farmers and collectors technique may help identify toxic mushrooms by detecting poisonous mushrooms using images of different mushroom species with distinctive morphological features. However, it can be not easy to distinguish between species. This paper used a dataset of 6714 mushroom images obtained from nine different mushroom species to classify the mushroom species. For a more straightforward comprehension of mushroom images and feature extraction by reanalysis of data sets, data visualization was performed using Grad-CAM, LIME, and Heatmap methods. Residual block-based Convolutional Neural Network (CNN) architectures are trained to automatically classify the concatenated feature map obtained from the Grad-CAM, LIME, and Heatmap methods. After extracting the deep features of the images from each architecture, the Atom Search Optimization (ASO) algorithm has been used to select the most distinctive features. The 6714×9000 size of the concatenated feature map was reduced to 6714×600 using the ASO algorithm. Classification results were evaluated using six different classifiers based on the feature map obtained to determine the mushroom species. The nine classes of mushroom species were classified successfully with 95.45 % accuracy using the proposed model with the ASO algorithm and KNN classifier. The methodology introduces novel visualization techniques for interpreting CNN-based models' decisions in mushroom species classification tasks. Using metaheuristics-based CNN models with multi-feature fusion techniques allows the model to leverage diverse sources of information, potentially enhancing its ability to discriminate between mushroom species and achieve higher classification accuracy than existing methods. This study can advance the mushroom species classification field by introducing new methodologies, improving classification accuracy, providing insights into model interpretability, and facilitating knowledge transfer to related fields. • The proposed model automatically detects mushroom species. • Grad-CAM, LIME, and Heatmap methods based on CNN architectures are used to visualize mushroom images. • The ASO algorithm has been successfully applied to residual block-based CNN models for feature selection. • The hybrid model classification accuracy for different species of mushrooms is 95.45 %. [ABSTRACT FROM AUTHOR]

Fall detection is an emerging problem in the field of elderly care and health monitoring. Falls can cause severe injuries and even death among the elderly, and prompt detection can significantly reduce the risk of further damage. Deep learning plays a pivotal role in fall detection by enabling the recognition of complex patterns and the analysis of spatial and temporal data, leading to accurate and efficient identification of fall events. However, some recent deep learning models still face challenges and limitations, particularly in terms of interpretability and explainability. In this paper, CABMNet has been proposed, an adaptive two-stage deep learning network that optimizes spatial features through convolutional neural networks (CNNs) and convolutional block attention module (CBAM) in the initial stage, followed by bidirectional long short-term memory (Bi-LSTMs) enhanced with multi-head attention (MHA) for temporal analysis in the subsequent stage. The incorporation of attention in the proposed work addresses issues including feature discrimination, temporal analysis, and model interpretability, leading to improved overall performance and efficiency of fall detection. Additionally, a Kalman filter is integrated to enhance noise reduction in the preprocessing stage. Evaluation on diverse five datasets indicates that CABMNet outperforms existing state-of-the-art models, achieving accuracy rates of 99.27% for KFall, 99.85% for MobiAct, 99.55% for SmartFall, 98.36% for URFall, and 98.52% for MobiFall. The proposed CABMNet demonstrates substantial applicability in healthcare, specifically in the domains of elderly care, health monitoring, and fall detection, contributing to enhanced well-being and safety for elderly people. • A two-stage model, CABMNet, integrates CNNs with CBAM and Bi-LSTMs with MHA. • CBAM with CNNs optimizes spatial feature extraction and computational efficiency. • BiLSTM uses MHA for precise recognition of crucial time steps and complex patterns. • The Kalman filter improves noise reduction and state estimation in preprocessing. • CABMNet is evaluated on KFall, MobiAct, SmartFall, URFall, and MobiFall datasets. [ABSTRACT FROM AUTHOR]

The estimation of respiratory rate (RR) is an extremely crucial step in the monitoring of a patient's health care. Nowadays, non-contact physiological signal measurement based on face videos has great potential for various applications. However, current RR estimation methods still face challenges in solving remote RR estimation tasks because the color change of facial skin is very subtle and the quasi-periodicity of the remote photoplethysmography(rPPG) requires the long-range temporal modeling. In this paper, we propose a dual-branch network that consists of an attention based convolutional neural network (CNN) and Transformer network (ACTNet), which can fully utilize local features and global information. In ACTNet, the CNN branch captures subtle color changes between facial image frames and introduces local attention to further highlight local features, and the Transformer branch models the long-term temporal relationships between video frame sequences to obtain global information. Then we fuse local features and global information into the network using the feature coupling unit. Comprehensive experimental results on COHFACE and DEAP datasets show that our method achieves state-of-the-art performance. • We propose an ACTNet consisting of attention based CNN and Transformer for RR estimation from facial videos, and it takes into account both global contextual information as well as local features. • Attention based CNN is used to enhance the local feature extraction ability which allows ACTNet to capture the subtle color changes from facial videos. • Experimental evaluation is performed on COHFACE and DEAP datasets, and ACTNet achieves state-of-the-art performance including Mean Absolute Error, Root Mean Square Error, Pearson Correlation Coefficient and generalization ability. [ABSTRACT FROM AUTHOR]

This paper proposes a new method for predicting downlink traffic volume in mobile networks, aiming to minimize overprovisioning while meeting specified service-level agreement (SLA) violation rates. We introduce a multivariate and multi-step prediction approach and compare four machine learning (ML) architectures: long short-term memory (LSTM), convolutional neural network (CNN), transformer, and light gradient-boosting machine (LightGBM). Our models predict up to 24 steps ahead and are evaluated under both single-step and multi-step conditions. Additionally, we propose parametric loss functions to adhere to SLA violation rate constraints. Our results emphasize the importance of using parametric loss functions to meet SLA constraints. We discovered that LSTM when paired with our custom multivariate feature sets, outperforms the transformer architecture in short-term forecasting up to 4 h ahead. For these short-term predictions, we demonstrate that methods based on domain knowledge, like our custom feature sets combined with simpler models such as LSTM, surpass more complex models like transformers. However, for long-term forecasting (8 to 24 h ahead), transformers outperform all other models. [ABSTRACT FROM AUTHOR]

• Noise-assisted data enhancement was used to overcome the poor picture quality in MSW sorting. • CNN and DRSN accuracies were boosted from 64.56 % and 70.30 % to 81.78 % and 90.32 %. • Noise-assisted data enhancement also relieved the over-fitting problem of CNN and DRSN models. • Detailed improvements were discussed by confusion matrix and ROC analysis. Data enhancement is an emerging approach to solve the limited sample size for computer vision-based municipal solid waste (MSW) classification. To overcome the unmatched features between real MSW pictures and those generated by data enhancement, this paper examined the potential of noise-assisted data enhancement methods on two representative MSW classification models: Convolutional Neural Networks (CNN) and Deep Residual Shrinkage Networks (DRSN). The experimental results showed that data enhancement significantly improved the model's accuracy compared with no data enhancement. Specifically, CNN model's test accuracy increased from 64.56 % to 81.78 %, while DRSN model rose from 70.30 % to 90.32 %. Notably, DRSN demonstrated exceptional robustness under noise conditions, achieving over 90 % accuracy in training and test sets. Analysis of the confusion matrix revealed the accuracy significantly improved in identifying glass, plastic, and metal under noise enhancement. These findings emphasized the importance of enhancement strategies in noisy industrial scenarios, offering guidance for MSW management. [Display omitted] [ABSTRACT FROM AUTHOR]

Recently, deep learning has achieved great success in visual tracking. The goal of this paper is to review the state-of-the-art tracking methods based on deep learning. First, we introduce the background of deep visual tracking, including the fundamental concepts of visual tracking and related deep learning algorithms. Second, we categorize the existing deep-learning-based trackers into three classes according to network structure, network function and network training. For each categorize, we explain its analysis of the network perspective and analyze papers in different categories. Then, we conduct extensive experiments to compare the representative methods on the popular OTB-100, TC-128 and VOT2015 benchmarks. Based on our observations, we conclude that: (1) The usage of the convolutional neural network (CNN) model could significantly improve the tracking performance. (2) The trackers using the convolutional neural network (CNN) model to distinguish the tracked object from its surrounding background could get more accurate results, while using the CNN model for template matching is usually faster. (3) The trackers with deep features perform much better than those with low-level hand-crafted features. (4) Deep features from different convolutional layers have different characteristics and the effective combination of them usually results in a more robust tracker. (5) The deep visual trackers using end-to-end networks usually perform better than the trackers merely using feature extraction networks. (6) For visual tracking, the most suitable network training method is to per-train networks with video information and online fine-tune them with subsequent observations. Finally, we summarize our manuscript and highlight our insights, and point out the further trends for deep visual tracking. [ABSTRACT FROM AUTHOR]

This paper studies the effect of word representations on gender classification using deep learning. There are two main objectives: how well do popular deep learning architectures, namely LSTM and CNNs, perform on gender classification task and investigate how the choice of word representation effects the performance. Three networks, LSTM, CNN and LeNet-5, were trained on a dataset containing about 18000 names from India, Western countries, Sri Lanka and Japan. These names, encoded using the popular One-Hot representation and Word Embeddings in addition to Integer representation and an Enhanced Integer representation (proposed in this paper), were given as Input and the performance is evaluated on accuracy, training times and size of input layer. Experimental results show that LSTM in combination with word embedding derived from the proposed Enhanced Integer representation gives the best performance of about 85%. One-Hot representation is superior to Integer and Enhanced Integer representation but appears to perform lower than word embeddings. [ABSTRACT FROM AUTHOR]

Diabetes mellitus, commonly known as diabetes, is a disease that affects a vast majority of people globally. Diabetes cannot be cured; it can only be kept under control. In this paper, diabetes is diagnosed by the analysis of Heart Rate Variability (HRV) signals obtained from ECG signals. We employed deep learning networks of Convolutional neural network (CNN) and CNN-LSTM (LSTM = Long short term memory) combination to automatically detect the abnormality. Unlike the conventional analysis methods so far followed, deep learning techniques do not require any feature extraction. We initially performed classification splitting the database into separate training and testing data. The maximum accuracy obtained for test data is 90.9% using CNN-LSTM. Using 5 fold cross-validation, CNN gave an accuracy of 93.6% while CNN-LSTM combination gave the maximum accuracy of 95.1%. As per our best knowledge, this is the first paper in which deep learning techniques are employed in distinguishing diabetes and normal HRV. The accuracy obtained using cross-validation is the maximum value achieved so far for the the automated detection of diabetes using HRV. [ABSTRACT FROM AUTHOR]

The primary purpose of this paper is to realize robust place recognition algorithms towards simultaneous viewpoint and condition changes, and provide satisfactory computational efficiency. In this paper, we significantly improve the viewpoint invariance of the SeqSLAM algorithm by using state-of-the-art deep learning techniques to generate robust feature representations of images and develop the SeqCNNSLAM . Experimental results show that SeqCNNSLAM outperforms state-of-the-art place recognition systems in most cases, such as, when precision is maintained at 100%, the maximum recall obtained by SeqCNNSLAM is 50% higher than SeqSLAM on the Norland dataset with simultaneous condition change and 12.5% viewpoint change. Besides, we develop an acceleration method called A-SeqCNNSLAM , which exploits the location relationship between the matching images of adjacent images to reduce the matching range of the current image. Experimental results demonstrate that an acceleration of  ∼  5 times is achieved with minimal accuracy degradation of  ∼  5%. Finally, to enable A-SeqCNNSLAM adaptability in new environments, O-SeqCNNSLAM is devised for the online parameter adjustment in A-SeqCNNSLAM. [ABSTRACT FROM AUTHOR]

With the significant increase in photovoltaic (PV) electricity generation, more attention has been given to PV power forecasting. Indeed, accurate forecasting allows power grid operators to better schedule and dispatch their assets, such as energy storage systems and reserve. In this paper, a hybrid deep learning model and a convolutional neural network with memory is proposed, to provide intraday (2 h) solar irradiance forecasts using sequentially-collected images from public webcams. The performance of the proposed model is compared to those of a standard time-series forecast models, a linear regression as well as state-of-the-art neural networks. All models are trained and tested using images collected from two webcams on EPFL's campus for just over a year. The results show that the proposed method outperforms all other models and matches the state-of-the-art methodology while providing simplicity of implementation and efficient computation. • A novel method for estimating global horizontal irradiance is proposed. • The method uses images public webcams instead of all-sky cameras. • The method combines images from different cameras to improve forecasts' accuracy. • The method can be extended to include other types of data (meteorological data). • This simple method matches state-of-the-art methods. [ABSTRACT FROM AUTHOR]

With the increasing impact of climate change and relative sea level rise, low-lying coastal communities face growing risks from recurrent nuisance flooding and storm tides. Thus, timely and reliable predictions of coastal water levels are critical to resilience in vulnerable coastal areas. Over the past decade, there has been increasing interest in utilizing machine learning (ML) based models for emulation and prediction of coastal water levels. However, flood advisory systems still rely on running computationally demanding hydrodynamic models. To alleviate the computational burden, these physics-based models are either run at small scales with high resolution or at large scales with low resolution. While ML-based models are very fast, they face challenges in terms of ensuring reliability and ability to capture any surge levels. In this paper, we develop a deep neural network for spatiotemporal prediction of water levels in coastal areas of the Chesapeake Bay in the U.S. Our model relies on data from numerical weather prediction models as the atmospheric input and astronomical tide levels, while its outputs are time series of predicted water levels at several tide gauge locations across the Chesapeake Bay. We utilized a CNN-LSTM setting as the architecture of the model. The CNN part extracts the features from a sequence of gridded wind fields and fuses its output to several independent LSTM units. The LSTM units concatenate the atmospheric features with respective astronomical tide levels and produce water level time series. The novel contribution of the present work is in spatiotemporality and in prioritization of the physical relationships in the model to maintain a high analogy to hydrodynamic modeling, either in the network architecture or in the selection of predictors and predictands. The results show that this setting yields a strong performance in predicting coastal water levels that cause flooding from minor to major levels. We also show that the model stands up successfully to the rigorous comparison with a high-fidelity ADCIRC model, yielding mean RMSE and correlation coefficient of 14.3 cm and 0.94, respectively, in two extreme cases, versus 12.30 cm and 0.96 for the ADCIRC model. The results highlight the practical feasibility of employing fast yet inexpensive data-driven models for resilient coastal management. • A deep learning model is developed to predict a wide range of coastal water levels using wind and tides as input. • CNN layers extract features from the wind field and fuse them to independent LSTM units, creating spatiotemporality. • The model compares well with the data for different storm scenarios. • The model is comparable to or surpasses a high-fidelity hydrodynamic model while taking a fraction of a second to run. [ABSTRACT FROM AUTHOR]

Lymphoma is a malignant tumor originating from the lymphohematopoietic system. At present, pathological evaluation is one of the important methods to diagnose malignant lymphoma. In clinical practice, the diagnosis of lymphoma, especially in newly diagnosed patients, depends mainly on histopathological examination of the lesion. The type of lymphoma is determined by repeatedly comparing hematoxylin-eosin (H&E) whole slide images (WSIs) and immunohistochemical WSIs under a microscope. It is a repetitive, tedious, and time-consuming process. Therefore, it is extremely important to establish a highly accurate and standardized lymphoma diagnosis algorithm. In this paper, we developed an innovative deep-learning framework based on multi-model fusion, which only uses the H&E slides, with special attention to gastric Mucosa-associated lymphoid tissue (MALT) lymphoma diagnosis. The proposed framework can evaluate and improve the auxiliary ability of the convolutional neural network (CNN) in clinical practice for the diagnosis of gastric MALT lymphoma. The proposed method achieved an accuracy of 98.53% using image patches and an accuracy of 94.96% on 258 WSIs. These results show the high accuracy in the diagnosis of MALT lymphoma and its potential use in clinical practice. In addition, we also estimated the 95% confidence interval of WSIs prediction values. The result shows that the proposed framework has a high degree of differentiation in the interpretation between gastric MALT lymphoma and normal pathological tissues. [ABSTRACT FROM AUTHOR]

As solar photovoltaic (PV) has emerged as a dominant player in the energy market, there has been an exponential surge in solar deployment and investment within this sector. With the rapid growth of solar energy adoption, accurate and efficient detection of PV panels has become crucial for effective solar energy mapping and planning. This paper presents the application of the Mask2Former model for segmenting PV panels from a diverse, multi-resolution dataset of satellite and aerial imagery. Our primary objective is to harness Mask2Former's deep learning capabilities to achieve precise segmentation of PV panels in real-world scenarios. We fine-tune the pre-existing Mask2Former model on a carefully curated multi-resolution dataset and a crowdsourced dataset of satellite and aerial images, showcasing its superiority over other deep learning models like U-Net and DeepLabv3+. Most notably, Mask2Former establishes a new state-of-the-art in semantic segmentation by achieving over 95% IoU scores. Our research contributes significantly to the advancement solar energy mapping and sets a benchmark for future studies in this field. • Transformer based deep learning model is introduced for PV panel segmentation in multi-resolution imagery. • Currently available open-source datasets for PV segmentation are summarized, including diverse multi-resolution datasets of satellite and aerial imagery. • The performance of state-of-art segmentation models, such as U-Net, DeepLabv3 and Mask2Former are evaluated in detail. [ABSTRACT FROM AUTHOR]

The rising cost and demand for energy have prompted several organizations to devise innovative methods for monitoring, managing, and conserving energy. Heat energy load prediction in smart buildings is critical for saving energy as well as for meeting energy demands. As the cost of energy and demand is increasing, people must find some way to control it and save energy. To overcome these issues, an intelligent energy management system needs to be developed to help individuals by predicting heat energy demand ahead of time and analyzing energy use patterns. This paper introduces a data-driven short-term heat energy consumption prediction for smart buildings by developing a hybrid deep learning model considering the capabilities of Convolutional and Recurrent Neural Networks. The proposed model extracts hidden patterns from actual data collected through sensors and also uses hand-crafted features to predict the heat energy demand accurately. The efficiency of the proposed model has been evaluated on real smart heat metering data, a case study in Denmark, having three kinds of buildings: single-family, apartment, and terraced houses. The extensive experimental results have been reported with real-world data by considering various evaluation metrics such as MAE, RMSE, MAPE and R 2. The results are promising and outperformed other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• A calculated method in space domain is presented to analyze the 1ω near-field status of SG-Ⅱlaser facility in batch, which can guide the operation and maintenance of the facility. • A residual convolutional neural network with attention mechanism is presented to identify near-field distribution features in a timely manner. • The spatial attention module (SA) enables the model to have the higher accuracy. • Methods can solve the problem that manual observation cannot detect laser near-field anomalies in time for large-scale science engineering. High-power solid-state laser facilities for Inertial confinement fusion impose very stringent requirements on the near-field distribution during operation. We propose calculated methods in space domain, including Otsu, adaptive rotation, cropping and numerical calculation to automatically process high power-laser near-field images from a large number of different status. These methods are applied to detect anomalous status based on contrast and modulation. It can also provide the dataset for the deep learning model. For characteristics of the 1ω near-field distributions due to various perturbations, a residual neural network with an added spatial attention mechanism is used to identify and classify features of numerous near-field images without human recognition. The accuracy of the model is close to 93%, which is better than the common residual neural network and VGG16 model. The results of this paper can be used to monitor the operation status and provide abnormal warning in the complex engineering of large-scale high-power laser facility based on timely analysis of the near-field status. [ABSTRACT FROM AUTHOR]

Grape leaf disease is a primary factor affecting grape yield and impeding the growth of the industrial economy. Currently, high-quality datasets are required for deep recognition models to effectively identify grape leaf diseases. In reality, the acquired image inevitably has the appearance of being low quality and unclear due to the influence of equipment and environment. The super-resolution method is an effective way to solve the problem. However, due to the complex model structure and learning strategies, existing super-resolution methods are computationally expensive and neglect the balance between global and local feature extraction. In this paper, a novel multiple attention transformer super-resolution (MATSR) method is proposed to solve these problems. Specifically, the method uses convolution to extract low-level features, which are then fed into the multiple attention shuffle transformer (MAST) module. The module consists of the shuffle transformer for global feature extraction, the constructed channel spatial attention block (CSAB) for local feature extraction, and the built reduisal feature block (RFB) for aggregating these two features. Meanwhile, considering the importance of the balance of global and local features for grape leaf disease recognition, a dynamic learning strategy is designed to balance these two features. Further, the abstract feature extraction layer combined the learned features with the low-level features. Finally, these abstracted features are fed into the pixels shuffle layer for super-resolution upsampling. Numerous comparative experiments are carried out on the collected high-resolution grape leaf disease images as well as the apple leaf pathological images. The results demonstrate the proposed method achieved the best PSNR of 38.152 dB and SSIM of 0.96 at scale ×2, and 35.896 dB and SSIM of 0.934 at scale ×3, respectively. Furthermore, four recognition models are experimented on the super-resolution grape disease dataset, among which ShuffleNet and Inception achieve the best accuracy of 98.28 % and 98.53 %, respectively. Therefore, this study can effectively aid grape growers in accurately identifying diseases and taking proactive measures. [ABSTRACT FROM AUTHOR]

• Automatic croup and pertussis cough sound classification method based on Channel attention and multiscale Mel-spectrogram. • Adaptive scale audio feature extraction to compute different scales of window sizes and generate the multiscale Mel-spectrogram. • The proposed method can attain comparatively good performance. Croup and pertussis are major illnesses that result in human fatality, especially in pediatric patients. Timely and accurate diagnosis of these diseases is crucial to reducing mortality rates. Therefore, there is a need for a low-cost, rapid, and accurate diagnostic solution. This paper proposes a croup and pertussis cough sound classification algorithm based on channel attention and multiscale Mel-spectrogram (CPCSC). Firstly, an automatic croup and pertussis cough classification method is implemented. Secondly, an adaptive scale audio feature extraction method (ASFE) is proposed, which is used to compute different scales of window sizes and hop length to generate the multiscale Mel-spectrogram (MSMel-spectrogram). Thirdly, a CNN model with a channel attention mechanism is proposed to extract features of the MSMel-spectrogram. The channel attention mechanism captures channel information to enhance model performance. Finally, the comparison results with six methods on the cough dataset, CSC4, demonstrate that CPCSC outperforms other comparison algorithms with an average accuracy, sensitivity, specificity, precision, and F1-score of 90.5%, 90.5%, 93.92%, 91.37%, and 90.25%, respectively. [ABSTRACT FROM AUTHOR]

Physiological signals, such as electrocardiogram (ECG) and wrist pulse signals (WPS), play an important role in diagnosing and preventing cardiovascular and other physiological diseases. Therefore, accurate classification of physiological signals has become the key to assist physicians in diagnosis. However, this field still faces several prominent challenges, including limited availability of data, imbalanced datasets, convergence issues with loss functions, and the need for model architectures capable of accurately detecting waveform patterns. This study introduces the Physiological Signal Classification Network (PSC-Net), which combines the strengths of Convolutional Neural Networks (CNNs) and transformers for applications in medical artificial intelligence. Specifically, local temporal features are extracted using the GRWA-LSTM network (GLNet) proposed in this paper. Within the transformer, two GRU layers replace the fully-connected layer to enhance global feature extraction for physiological signal data. Residual connection integrates the outputs of GLNet and Transformer through global average pooling and weight settings. To address challenges related to small and imbalanced datasets, we propose an enhanced data augmentation algorithm based on SMOTE Tomek, along with an improved loss function. Additionally, automatic learning rates are optimized using the Dung Beetle Algorithm (DBA). Our proposed method achieves superior accuracies of 83.33%, 100.0%, 95.74%, and 98.85% on four physiological signal datasets (including one clinical dataset): Five Types of Pulses Database, Coronary Heart Disease (CHD) database, MIT-BIH Arrhythmia Database, and MIT-BIH ST Change Database. These results attest to the model's robust generalization capability and its promising application prospects in assisting diagnoses. [ABSTRACT FROM AUTHOR]

Wafer maps include information about multiple defect patterns on the wafer surface. Intelligent categorization of the defective wafer is essential for investigating the underlying causes and improving the reliability and safety of the entire system. Recently, convolutional neural networks (CNNs) have been widely employed to construct successful defect detectors by learning from offline defect datasets. However, traditional CNN-based detectors are costly and incapable of unknown production defect detection despite the accurate performance. In this paper, we propose a novel IL-based method, called PIRB, for online unknown wafer defect detection. Specifically, we leverage the neural networks to remember old defect patterns by selectively restricting learning on the important weights. A tiny reference buffer is applied to preserve the experienced wafer defect patterns in the learning process to facilitate the detection accuracy. The experimental results show that the proposed method works well for classifying unknown defects, with a 60% reduction in training time compared to offline learning and a 10% increase in total accuracy compared to the state-of-the-art methods. • A novel framework for unknown wafer defect detection and reliability analysis. • A strategy Permutation-Mix is introduced to reduce training consumption. • Demonstrate the feasibility of the proposed method by comparing with various methods. [ABSTRACT FROM AUTHOR]

Although the dangers of working in confined spaces have been known for many years, fatal accidents related to working in confined spaces still frequently occur. Considerable research has been conducted to identify potential contributory factors of confined space incidents through analyzing accident reports. However, accident databases are usually read and interpreted manually by human experts. The process of analyzing confined space accident reports can be time-consuming and labor-intensive. As the number of accident records increases, it is difficult for the experts to manually review all the reports. Moreover, different individuals may reach various conclusions from the same accident report. Some analysts may fail to capture all the meaningful and relevant causal factors. Automatic information extraction using special rules and ontology-based approaches can be used to mine reports of confined space accidents. However, such approaches tend to suffer from the problem of weak generalization. To overcome this limitation and improve the performance of contributory factors analysis, an improved deep learning based framework is proposed in this paper to automatically extract and classify contributory factors from confined space accident reports using BERT-BiLSTM-CRF and CNN models. Research results suggested that the proposed framework can be used as a feasible method to qualitatively and quantitatively explore the contributory factors of confined space accidents. By analyzing a large quantity of confined space accident reports, the frequency of contributory factors can be estimated automatically. This outcome is helpful to significantly improve the risk assessment quality of confined space works. [ABSTRACT FROM AUTHOR]

In this paper we introduce a biologically inspired Convolutional Neural Network (CNN) architecture called LGN-CNN that has a first convolutional layer composed of a single filter that mimics the role of the Lateral Geniculate Nucleus (LGN). The first layer of the neural network shows a rotational symmetric pattern justified by the structure of the net itself that turns up to be an approximation of a Laplacian of Gaussian (LoG). The latter function is in turn a good approximation of the receptive field profiles (RFPs) of the cells in the LGN. The analogy with the visual system is established, emerging directly from the architecture of the neural network. A proof of rotation invariance of the first layer is given on a fixed LGN-CNN architecture and the computational results are shown. Thus, contrast invariance capability of the LGN-CNN is investigated and a comparison between the Retinex effects of the first layer of LGN-CNN and the Retinex effects of a LoG is provided on different images. A statistical study is done on the filters of the second convolutional layer with respect to biological data. In conclusion, the model we have introduced approximates well the RFPs of both LGN and V1 attaining similar behavior as regards long range connections of LGN cells that show Retinex effects. [ABSTRACT FROM AUTHOR]

In humans, Congestive Heart Failure (CHF) refers to the chronic progressive condition that drastically influences the pumping potentiality of the heart muscle. This CHF has the possibility of increasing health expenditure, morbidity, mortality and minimized quality of life. In this context, Electrocardiogram (ECG) is considered as the simplest and a non-invasive diagnosis method that aids in detecting and demonstrating the realizable changes in CHF. However, diagnosing CHF based on manual exploration of ECG signals is frequently impacted by errors as duration and small amplitude of the signals either investigated separately or in the integration is determined to neither specific nor sensitive. At this juncture, the reliability and diagnostic objectivity of ECG signals during the CHF detection process may be enhanced through the inclusion of automated computer-aided system. In this paper, Deep CNN and LSTM Architecture (DCNN-LSTM)-based automated diagnosis system is proposed for detecting CHF using ECG signals. In specific, CNN is included for the purpose of extracting deep features and LSTM is used for attaining the objective of CHF detection using the extracted features. This proposed DCNN-LSTM is evolved with minimal pre-processing of ECG signals and does not involve any classification process or manual engineered features during diagnosis. The experimentation of the proposed DCNN-LSTM conducted using the real time ECG signals datasets confirmed an accuracy of 99.52, sensitivity of 99.31%, specificity of 99.28%, F-Score of 98.94% and AUC of 99.9%, respectively. [ABSTRACT FROM AUTHOR]

• A hybrid CNN and Bi-LSTM based EMGHandNet architecture is successfully demonstrated for the classification of human hand activities using the sEMG signals. • The proposed method exploits the learning of the inter-channel relations using the 1-D convolutional layers and two way temporal features with the Bi-LSTM layers. • We have performed experiments on five benchmark sEMG datasets namely the NinaPro DB1, the NinaPro DB2, the NinaPro DB4, the BioPatRec DB2, and the UCI Gesture. • The proposed EMGHandNet shows improved classification accuracy over the state of the art methods. • The impact of different input shapes on classification performance is analyzed. Recently, Convolutional Neural Networks (CNNs) have been used for the classification of hand activities from surface Electromyography (sEMG) signals. However, sEMG signal has spatial sparsity due to position of electrodes on hand muscle and temporal dependency due to performance of activity over a period of time. The CNN has the ability to extract spatial features and is limited in extracting temporal dependencies. Whereas, the Long Short-Term Memory (LSTM) aims to encode the temporal relations from sequential data. Hence, in this paper, we propose a hybrid CNN and Bidirectional LSTM (Bi-LSTM) based EMGHandNet architecture to encode the inter-channel and temporal dependencies of sEMG signals for hand activity classification. First, the CNN layers are used to extract deep features from sEMG signals, then these feature maps are processed by the Bi-LSTM to extract the sequential information in both the forward and backward directions. Thus, the proposed model learns both inter-channel and bidirectional temporal information in an end-to-end manner. The proposed model is trained and tested on five benchmark datasets, including the NinaPro DB1, NinaPro DB2, NinaPro DB4, BioPatRec DB2 and UCI Gesture. The average classification accuracies for the NinaPro DB1, NinaPro DB2, NinaPro DB4 and UCI Gesture are 95.77 % , 95.9 % , 91.65 % , and 98.33 % respectively. They correspond to an improvement of 4.42 % , 12.2 % , 18.65 % and 1.33 % over the respective state-of-the-art models. Moreover, for the BioPatRec DB2 dataset, a comparable performance (91.29 %) is observed. The experimental results and comparisons confirm the superiority of the proposed model for hand activity classification from the sEMG signals. [ABSTRACT FROM AUTHOR]

Rapid urbanization impacts water quality because contaminants from the urban environment accumulate in the water and pollute it and because there is more rivalry for water among municipalities, businesses, and other sectors such as farming. A change in the microclimate, fluid mechanics, geomorphic, ecological, or biogeochemical conditions will impact the water's quantity and quality. There is a reduction in the groundwater because of the difficulty that water has soaked into the earth as more roads are built. When the rain washes over impervious buildings like roadways and roofs, it leaves excessive pollution in water bodies. Both people and aquatic life may be at risk from the increased water pollution. This paper uses deep learning methods such as Convolutional Neural Networks (CNN) and Long-Short Term Memory (LSTM) to classify water quality. Next, it identifies the air quality in Urban Development (Conv. LSTM). The convolutional LSTMs use convolutional layers and the recurrent connections found in LSTMs. This allows the model to capture spatial dependencies in the input data in addition to the temporal dependencies captured by the recurrent connections. We also use thorough exploratory analysis to investigate the various beach habitats and the kinds of trash discovered in multiple places. Lowering water pollution and raising air quality are both strategies that can be employed to ensure sustainable urban development. The performance metrics such as accuracy, recall, precision, and F1-score are evaluated and classify the water pollution efficiently. In the water pollution dataset, the algorithms of RNN 65%, DBN 78%, LSTM 82%, and the proposed work of Conv.LSTM 92%. Similarly, for the air pollution dataset, the algorithms of RNN 60%, DBN 75%, LSTM 80%, and the proposed work of Conv.LSTM 91%. • This paper utilizes CNN and LSTM for water quality classification using deep learning. • Model captures spatial dependencies in input data, along with temporal dependencies from recurrent connections. • Thorough exploratory analysis investigates beach habitats and trash types found in multiple locations. • Evaluated performance metrics (accuracy, recall, precision, F1-score) efficiently classify water pollution. [ABSTRACT FROM AUTHOR]

• A 7-layer lightweight plant disease identification network LiteCNN is designed and knowledge distillation method is used to train LiteCNN, so that LiteCNN has a simple structure as well as high accuracy. • Separable convolution instead of regular convolution is employed to reduce the computation and the number of parameters of LiteCNN. • A low-redundancy block convolution approach is designed to reduce repeated data reading, thus reducing the latency of LiteCNN circuit. • BN layer is fused into the previous convolutional layer or fully-connect layer, instead of designing a standalone modular circuit for BN layer, so as to reduce the resource consumption and computation time. • The acceleration circuit of LiteCNN is designed and implemented on the FPGA development board ZYNQ Z7-Lite 7020, and is optimized for its parallel computation with four methods: unrolling the for-loop, pipelining the for-loop, loop_flattening and array partitioning. • LiteCNN is implemented on different devices, and the performance are compared in terms of speed, accuracy, power consumption and price. Using convolutional neural network (CNN) to identify plant diseases in-situ is a hot research topic in smart agriculture. Due to the memory-intensive and compute-intensive characteristics of CNN algorithm, it is difficult to implement CNN on edge terminals with limited memory and computational resources. In this paper, Field Programmable Gate Array (FPGA) is used to accelerate CNN to identify plant diseases. First, a 7-layer simple-structured network called "LiteCNN", with only 176 K parameters and 78.47 M floating point operations (FLOPs) was designed. And knowledge distillation method was used to train LiteCNN, making that the accuracy reaches 95.24 %. Secondly, the acceleration circuit of LiteCNN was designed and implemented on "ZYNQ Z7-Lite 7020″ FPGA board. To compress the network and speed up plant disease identification, the following methods were applied: 1) Separable convolution took place of regular convolution, and a low-redundancy block convolution approach was used to load data; 2) The Batch Normalization (BN) layer was fused into the previous convolutional layer (or fully-connected layer); 3) Feature data and model parameters were expressed by half float data. As the basic function of the circuit achieved, methods including unrolling the for-loop, pipelining the for-loop, loop flattening and array partitioning were used to optimize the parallelism of the circuit. Finally, LiteCNN on the FPGA board was verified. The plant disease identification accuracy was 95.71 %, the inference speed was 0.071 s per frame, and the power consumption was 2.41 W. The results show that this paper proposed a low-power, high-accuracy and fast-speed plant disease identification terminal, which can be well applied for real-time plant disease identification in the field. [ABSTRACT FROM AUTHOR]

The peak period of an energy-generating wave is one of the most important parameters that describe the spectral shape of the oceanic wave, as this indicates the duration for which the waves prevail with respect to their maximum extractable energy. In this paper, a half-hourly peak wave energy period (T P) forecast model is constructed using a suite of statistically significant lagged inputs based on the partial auto-correlation function with an extreme learning machine model developed and its predictive utility is benchmarked against deep learning models, i.e., convolutional neural network (CNN/CovNet) and recurrent neural network (RNN) models and other traditional M5tree, Conditional Maximization based Multiple Linear Regression (MLR-ECM) and MLR models. The objective model (ELM) vs. the comparison models (CNN, RNN, M5tree, MLR-ECM, and MLR) were trained and validated independently on the test dataset obtained from coastal zones of eastern Australia that have a high potential for implementation of wave energy generation systems. The outcomes ascertain that the ELM model can generate significantly accurate predictions of the half-hourly peak wave energy period, providing a good level of accuracy relative to deep learning models in selected coastal study zones. The study establishes the practical usefulness of the ELM model as being a noteworthy methodology for the applications in renewable and sustainable energy resource management systems. • A real-time peak wave energy period forecasting model is constructed using ELM model. • The advance ELM is benchmarked against a deep learning and other traditional models. • The advance ELM generates better predictions than deep learning models in this region. • ELM model provides significant energy management implications for ocean waves. [ABSTRACT FROM AUTHOR]

For the object-based classification of high resolution remote sensing images, many people expect that introducing deep learning methods can improve then classification accuracy. Unfortunately, the input shape for deep neural networks (DNNs) is usually rectangular, whereas the shapes of the segments output by segmentation methods are usually according to the corresponding ground objects; this inconsistency can lead to confusion among different types of heterogeneous content when a DNN processes a segment. Currently, most object-based methods utilizing convolutional neural networks (CNNs) adopt additional models to overcome the detrimental influence of such heterogeneous content; however, these heterogeneity suppression mechanisms introduce additional complexity into the whole classification process, and these methods are usually unstable and difficult to use in real applications. To address the above problems, this paper proposes a simplified object-based deep neural network (SO-DNN) for very high resolution remote sensing image classification. In SO-DNN, a new segment category label inference method is introduced, in which a deep semantic segmentation neural network (DSSNN) is used as the classification model instead of a traditional CNN. Since the DSSNN can obtain a category label for each pixel in the input image patch, different types of content are not mixed together; therefore, SO-DNN does not require an additional heterogeneity suppression mechanism. Moreover, SO-DNN includes a sample information optimization method that allows the DSSNN model to be trained using only pixel-based training samples. Because only a single model is used and only a pixel-based training set is needed, the whole classification process of SO-DNN is relatively simple and direct. In experiments, we use very high-resolution aerial images from Vaihingen and Potsdam from the ISPRS WG II/4 dataset as test data and compare SO-DNN with 6 traditional methods: O-MLP, O+CNN, OHSF-CNN, 2-CNN, JDL and U-Net. Compared with the best-performing method among these traditional methods, the classification accuracy of SO-DNN is improved by up to 7.71% and 10.78% for single images from Vaihingen and Potsdam, respectively, and the average classification accuracy is improved by 2.46% and 2.91% for the Vaihingen and Potsdam images, respectively. SO-DNN relies on fewer models and easier-to-obtain samples than traditional methods, and its stable performance makes SO-DNN more valuable for practical applications. [ABSTRACT FROM AUTHOR]

Network pruning has been widely used in the field of model compression and inference acceleration for convolutional neural networks(CNN). Existing methods generally follow a "training-pruning-retraining" paradigm, known as a three-stage pipeline. However, it cannot play an effective role in a pre-trained model with a larger pruning rate. In addition, prevailing methods usually set pruning rates as super parameters, which fail to consider the sensitivity of different convolution layers. In this paper, a novel pruning approach, based on the separation of sparsity search and model training(SST), is proposed to solve the above problems. Specifically, an evolutionary algorithm is introduced into the process of searching for the most suitable number of pruned filters for every layer. After obtaining the best sparsity structure, a new pruning strategy, called the one-pruning pipeline, is utilized to prune the pre-trained model. Experiments on multiple advanced CNN architectures show that SST can greatly improve the pruning rate with a slight loss of accuracy, which is found to universally reduce more than 60% FLOPs on CIFAR-10. Notably, on ILSVRC-2012, pruning based on ResNet18 reduces FLOPs by 42.8%, while top-1 and top-5 accuracy only lose 1.19% and 0.62%, respectively. [ABSTRACT FROM AUTHOR]

• Strengthen the descriptor by effectively combing features at different levels of CNN. • Capture the spatial information of a local patch with spatial context pyramid. • Comparable performance with state-of-the-art and comprehensive ablation experiments. Despite that efforts have shifted to learning local descriptors with convolutional neural network (CNN) from hand-crafted realm, the inherent feature hierarchy within CNN has been rarely explored. To increase both the invariant and discriminative abilities of the CNN-based local descriptors by making use of the complementary representation powers of the feature maps at different levels of CNN, in this paper, we design a multi-level feature aggregation (MLFA) module to communicate information across pyramid levels effectively. Then, each level extracts a feature vector after feature fusion and the final descriptor concatenates these outputs. Moreover, to leverage the spatial structure within a local patch, we propose a novel spatial context pyramid (SCP) module to capture the spatial information. SCP is devised in a residual manner and only several additional parameters are introduced to the model. We implement our algorithm based on the HardNet framework and carry out comprehensive evaluation on the UBC Phototour, HPatches and ETH datasets. The experimental results demonstrate that the proposed method performs favorably against the state-of-the-art ones. Ablation study is also provided to show the effectiveness of each component. [ABSTRACT FROM AUTHOR]

Lossy image compression is likely to produce blurred images which leads to erroneous image-level understanding. The human visual system (HVS) is focused on the area of interest present in the image. Motivated by this fact, we propose a compression-decompression algorithm focusing on the important content present in different parts of the image. Using the concept of the Guided Grad-CAM (Gradient-weighted Class Activation Mapping) technique to produce heat maps with the help of a heat map generator trained on ResNet-50, a saliency-guided encoding–decoding algorithm is developed. A wider multi-scale saliency guided convolutional neural network (MsSG-CNN) is designed, in which the notion of convolution with different size filters helps to obtain unique but different features. The feature extraction followed by multilevel fusion of features helps deep neural network (DNN) to capture contextual information and obtain high quality, good resolution, visually pleasing images with fine details. The proposed algorithm is tested on the Kodak benchmark dataset, CLIC 2019 challenging dataset, and FDDB facial images dataset. At low bit rates, the MS-SSIM of the proposed algorithm is found to be superior to JPEG, JPEG2000, BPG, WebP, and Minnen's approaches with approximately up to 60%, 24.80%, 11.43%, 23.08% & 75% gains respectively, which is quite a significant improvement, when tested on the Kodak dataset. Similarly, at high bit rates, the improvement in MS-SSIM is approximately up to 41.67%, 37.30%, 23.90% 34.21% & 13.33% when compared with JPEG, JPEG2000, BPG, WebP, and Minnen's approaches respectively. The improvement in PSNR at low and high bit rates is approximately up to 11.32%, 5%, 5.26% and 5.6%, 10.29%, 8.7% as compared to JPEG, Balle's, and Lee's algorithms respectively. The PSNR-HVS has been improved by approximately up to 27.27%, 19.15%, and 28.33%, 28% as compared to JPEG and Toderici's algorithms respectively at low and high bit-rates. A similar type of improvement is obtained with FDDB and CLIC 2019 datasets also, which is discussed in the paper. [ABSTRACT FROM AUTHOR]

Stand-alone and the combined model of Convolutional Neural networks (CNNs) and Long Short-Term Memory (LSTM) and Deep neural Networks (DNNs) have shown great improvements in a variety of Speech Recognition tasks. In this paper we also combined these networks but in this paper we used them for multilingual speech recognition, for the prediction and correction (PAC) architecture, in order to calculate the state probability. Our proposed model is known as PAC-MCLDNN. In this paper, we present experiment results for multilingual training on AP16-OLR task. Furthermore, cross-lingual model transfer and multitask learning for under resourced languages such as Uyghur and Vietnam are also performed which further improved the recognition results. [ABSTRACT FROM AUTHOR]

In the past few years, convolutional neural networks (CNN) and Transformer have achieved great success in medical image segmentation, but they each have inevitable drawbacks. Among them, convolution operation is difficult to calculate the relationship between elements at a certain position and elements far away from the position in the feature map. However, Transformer tends to ignore the importance of local information when exploring the correlation between overall elements. In order to allow the network to acquire both the ability to explore local details and compute the correlation between distant elements, this paper proposes TransUNet++ based on TransUNet. Specifically, this paper proposes two modules, Big Model and Small Model, to explore the element relationship between feature maps. Among them, on the basis of the whole feature map as the basic unit, the Big model can not only calculate the correlation between distant elements in the feature map but also extract the detailed information of the local feature map. On the basis of taking 1/4 of the feature map as the basic unit, the Small model not only explores the correlation between distant elements but also extracts the local details of the feature map. We demonstrate on the Synapse multi-organ segmentation dataset(Synapse) and Automated cardiac diagnosis challenge dataset (ACDC) that using either the Big Model or the Small Model alone can improve the experimental results, and using the Big Model and the Small Model in parallel can achieve more optimal experimental results. Among them, in Synapse dataset, we achieved 80.87% dice score and 24.79% HD score, and in ACDC dataset, we achieved 91.41% dice score and 1.08% HD score. [Display omitted] • The Big Model can be used for remote modeling and local information extraction. • The Small models can be used for mid-range modeling and local detail extraction. • The advantages and disadvantages of the convolution kernel and Transformer by talked. [ABSTRACT FROM AUTHOR]

Collaborative Robots need to read the surfaces they are walking on to keep their dynamic equilibrium, regardless of whether the ground is flat or uneven. Although accelerometers are frequently employed for this task, previous efforts have centered on retrofitting the quadruped robots with new sensors. The second technique is to collect lots of samples for machine learning algorithms, which are not widely implemented. Learning-based approaches altered the traditional way of data analytics. The advanced deep learning algorithms provide better accuracy and prove more efficient when the data size is large. This paper introduced a novel architecture of Convolutional Neural Network, a deep learning-based approach for efficiently classifying the surface on which the robots are walking. The dataset contains reading captured by Inertia Measurement Unit sensors. The proposed model achieved an overall classification accuracy of 88%. The proposed architecture is compared with the existing deep and machine learning techniques to show its effectiveness. The proposed model can be installed on collaborative robots' onboard processors to identify the surfaces effectively. • Cobots must read the surfaces they are walking on to maintain dynamic balance. • The paper presents a novel CNN architecture for efficient surface detection. • Current machine/deep learning architectures are compared against the proposed model. • The proposed model achieved overall classification accuracy of 88%. [ABSTRACT FROM AUTHOR]

• This paper proposes the application of deep learning algorithms for diagnosing 14 major ophthalmological defects such as Hollenhorst Emboli, Arteriosclerotic Retinopathy etc. • In this study, multiple performance evaluation techniques such as Precision, Recall, F-1 Score, etc. are used to compare deep learning algorithms. • In this study, the performance compared with the existing literature achieved higher accuracy due to the unique model and its configuration, hyperparameter tuning and pre-processing techniques for the 14 classes of retinal defects. Retina is the heart of an eye which generates electrical impulses due to light sensitivity. The vessel formation in human eye is an essential key for diagnosing ophthalmological conditions. This paper aims to diagnose ophthalmological conditions through deep learning models and provide advancements in early detection of ophthalmological conditions for proper treatment to protect patient's vision, and for health care giver worldwide. STARE dataset is used for this study which consists over 385 retinal images of 14 ophthalmological defects such as BRAO, CRAO, etc. This dataset is further pre-processed over the techniques such as augmentation, normalization, etc for obtaining the best refined features for training deep learning algorithms. This paper broadly implements 5 deep learning algorithms i.e., EfficientNet, 3-Layers CNN, InceptionV2, ResNet-50, VGG-16. These models are trained number of times over tuned hyperparameters such as batch size etc and evaluated over 4 performance metrics over weighted averaged and macro averaged of precision, recall, F1-score, and accuracy to acquire the best performing model. EfficientNet performed the best with 98.43% accuracy, macro averaged f-1 score, recall, precision as 98.37%, 99.16%, 97.91% and weighted averaged f-1 score, recall, precision, as 98.50%, 98.43%, 98.82% over batch size 64. In this study, the performance compared with the existing literature achieved higher accuracy due to the unique model and its configuration, hyperparameter tuning and pre-processing techniques for the 14 classes of retinal defects. The future work includes classifying more ophthalmological conditions, adding more parameters from blood, etc. [ABSTRACT FROM AUTHOR]

Cross-modal crowd counting aims to use the information between different modalities to generate crowd density images, so as to estimate the number of pedestrians more accurately in unconstrained scenes. Due to the huge differences between different modal images, how to effectively fuse the information between different modalities is still a challenging problem. To address this problem, we propose a cross-modal crowd counting method based on CNN and novel cross-modal transformer, which effectively fuses the information between different modalities and boosts the accuracy of crowd counting in unconstrained scenes. Concretely, we first design double CNN branches to capture the modality-specific features of images. After that, we design a novel cross-modal transformer to extract cross-modal global features from the modality-specific features. Furthermore, we a propose cross layer connection structure to connect the front-end information and back-end information of the network by adding different layer features. At the end of the network, we develop a cross- modal attention module to strengthen the cross-modal feature representation by extracting the complementarities between different modal features. The experimental results show that the method combining CNN and novel cross-modal transformer proposed in this paper achieves state-of-the-art performance, which not only effectively improves the accuracy and robustness of cross-modal crowd counting, but also has good generalization under multimodal crowd counting. • This paper is the first time to use the combination of CNN and Transformer in cross-modal crowd counting. • Complementary information between cross-modal images is important for cross-modal crowd counting. • Large receptive field is helpful for accurate counting results. [ABSTRACT FROM AUTHOR]

People counting is one of the key techniques in video surveillance. This task usually encounters many challenges in crowded environment, such as heavy occlusion, low resolution, imaging viewpoint variability, etc. Motivated by the success of R-CNN (Girshick et al., 2014) [1] on object detection, in this paper we propose a head detection based people counting method combining the Adaboost algorithm and the CNN. Unlike the R-CNN which uses the general object proposals as the inputs of CNN, our method uses the cascade Adaboost algorithm to obtain the head region proposals for CNN, which can greatly reduce the following classification time. Resorting to the strong ability of feature learning of the CNN, it is used as a feature extractor in this paper, instead of as a classifier as its commonly-used strategy. The final classification is done by a linear SVM classifier trained on the features extracted using the CNN feature extractor. Finally, the prior knowledge can be applied to post-process the detection results to increase the precision of head detection and the people count is obtained by counting the head detection results. A real classroom surveillance dataset is used to evaluate the proposed method and experimental results show that this method has good performance and outperforms the baseline methods, including deformable part model and cascade Adaboost methods. [ABSTRACT FROM AUTHOR]

In recent years, convolutional neural networks (CNNs) have been successfully applied in the field of image processing, and have been deployed to a variety of artificial intelligence systems. However, such neural models are still considered to be "black box" for most tasks. Two of fundamental issues underlying this problem are as follows: 1. What type of knowledge learned by a neural network in a task was not understandable and predictable? 2. The decision made by a neural model was generally not evaluable. Like neural coding in the brain, some neurons only participated in encoding a particular task. Inspired by this, in this paper, we propose a method to modify traditional CNN models into understandable CNNs, to clarify the information coding in high conv-layers of CNNs and further evaluate the decisions made by a neural model. In our understandable CNN models, each neural node (feature map) in a selected conv-layer was assigned to participate in encoding only one class in a classification task. Our models use the same training data as ordinary models without the need for additional annotations for supervision. We applied our method to the ResNet and DenseNet models. The experiments showed that new models can learn the information coding mode that we expected in an image-recognition task, and, using the pre-assigned coding mode, we can interpret why a neural model makes a right or wrong decision, which decisions are credible, and which are not. [ABSTRACT FROM AUTHOR]

The non-invasive electroencephalogram (EEG) signals enable humans to communicate with devices and have control over them, this process requires precise classification and identification of those signals. The recent revolution of deep learning has empowered both feature extraction and classification in a joint manner of different data types. However, deep learning is a data learning approach that demands a large number of training samples. Whilst, the EEG research field lacks a large amount of data which restricts the use of deep learning within this field. This paper proposes a novel augmentation method for enlarging EEG datasets. Our CutCat augmentation method generates trials from inter- and intra-subjects and trials. The method relies on cutting a specific period from an EEG trial and concatenating it with a period from another trial from the same subject or different subjects. The method has been tested on shallow and deep convolutional neural networks (CNN) for the classification of motor imagery (MI) EEG data. Two input formulation types images and time-series have been used as input to the neural networks. Short-time Fourier transform (STFT) is used for generating training images from the time-series signals. The experimental results demonstrate that the proposed augmentation method is a promising strategy for handling the classification of small-scale datasets. Classification results on two EEG datasets show advancement in comparison with the results of state-of-the-art researches. [ABSTRACT FROM AUTHOR]

Emotional dependencies play a crucial role in understanding complexities of dyadic interactions. Recent studies have shown that affect recognition tasks can benefit by the incorporation of a particular interaction's context, however, the investigation of affect context in dyadic settings using neural network frameworks remains a complex and open problem. In this paper, we formulate the concept of dyadic affect context (DAC) and propose convolutional neural network (CNN) based architectures to model and incorporate DAC to improve continuous emotion recognition (CER) in dyadic scenarios. We begin by defining a CNN architecture for single-subject CER-based on speech and body motion data. We then introduce dyadic CER as a two-stage regression framework. Specifically, we propose two dyadic CNN architectures where cross-speaker affect contribution to the CER task is achieved by: (i) the fusion of cross-subject affect (FoA) or (ii) the fusion of cross-subject feature maps (FoM). Based on the preceding dyadic models, we finally propose a new Convolutional LSTM (ConvLSTM) model for the dyadic CER. ConvLSTM architecture captures local spectro-temporal correlations in speech and body motion as well as the long-term affect inter-dependencies between subjects. Our multimodal analysis demonstrates that modeling and incorporation of the DAC in the proposed CER models provide significant performance improvements on the USC CreativeIT database and the achieved results compare favorably to the state-of-the-art. • Two-stage dyadic continuous emotion recognition framework for realistic scenarios • Novel CNN-based affect and feature-map fusion models in dyadic settings • Novel ConvLSTM architecture in dyadic settings to exploit dyadic affect context [ABSTRACT FROM AUTHOR]

This paper aims to extend collective interpretation to networks with complicated components. The collective interpretation is used to generate an internally interpretable model independently of specific inputs and learning conditions. The internally interpretable model is obtained by network compression where multi-layers are sequentially compressed, taking into account all possible routes from inputs to outputs. The network compression is easily applied to fully connected networks, but it cannot be applied to some networks with complicated components. Thus, to make the compression possible, we black-box partially and minimally these components to be replaced by the ordinary components. For demonstrating the effectiveness of this technique, we use here a new model based on the self-organizing map (SOM). Then, we introduce the convolutional neural networks (CNN) for dealing with SOM knowledge, usually represented in two-dimensional lattices. Because our network compression cannot deal with those convolutional components, we temporarily black-box the CNN components. Fixing the other connection weights, we re-train the partially black-boxed network to obtain the simplest prototype model for interpretation. The method was applied to two well-known data sets, and we demonstrated that the present method could compress the networks to get the simplest and interpretable ones. In addition, very stable compressed weights for interpretation could be obtained for easy interpretation. The results suggest that the main mechanism of multi-layered neural networks is based on linear relations between individual inputs and targets, to which peripheral non-linear ones are added. [ABSTRACT FROM AUTHOR]

In the paper, the automated evolutionary approach FEDOT-NAS for the design of convolutional neural networks is proposed. It allows building object recognition models for remote sensing tasks. The comparison of the proposed approach with state-of-the-art tools for neural architecture search is conducted for several examples of satellite-related datasets. The results of the experiments confirm the correctness and effectiveness of the proposed approach. The implementation of FEDOT-NAS is available as an open-source tool. [ABSTRACT FROM AUTHOR]

The pandemic of Covid-19 has caused a shift of paradigm of education, from face-to-face to e-learning. E-learning leads to an escalation in digitalization of handwritten documents because it requires submission of homework and assignments through online. To help teachers in checking digitalized handwritten homework, this paper proposes an automatic checking system based on a convolutional neural network (CNN) for handwritten numeral recognition. The CNN is used to recognize four arithmetic operations in mathematical questions consisting of addition, deduction, multiplication and division. The performance CNN in handwritten numeral recognition have been optimized in terms of activation function and gradient descent algorithm. The proposed CNN is also trained and tested with the MNIST handwritten data set. The experimental results show that the recognition accuracy the improved CNN improves to a certain extent as compared to before optimization. [ABSTRACT FROM AUTHOR]

Epilepsy is a chronic disease caused by sudden abnormal discharges of neurons which lead to transient brain disfunction. For those patients who need surgery, the recognition of the epileptic foci is one of the most important steps. In this paper, a combination of EEG signals processed by Short-time Fourier Transform and processed by Continuous Wavelet Transform is introduced. A deep learning model with an accuracy of 91.3% trained by Convolution Neural Network has been obtained. The experiment uses a methodology that has not been widely tested so far, and it also demonstrated the practical values by its relatively small time consumption and computing power requirements meanwhile ensuring high accuracy. Furthermore, it proves the feasibility of improving accuracy by combining different feature extraction methods. [ABSTRACT FROM AUTHOR]

This paper proposes a deep learning sequence-to-sequence approach to improve the task of automatic Romanian lemmatization. The study compares 24 systems using different combinations of recurrent, convolutional and attention layers, while the text input consists of word-lemma pairs, both at word and trigram level. As Romanian is a low resourced language in the field of text processing, the aim of this study is to use as little input information as possible. Thus, to increase the lemmatization accuracy, additional lexical features (such as context and POS tags) have been provided only gradually. For the trigrams case, two scenarios were proposed: to predict the lemma for every word from the sequence or to predict the lemma only for the word in the middle. The lemmatizers have been analyzed on two Romanian datasets: the Romanian Explicative Dictionary (DEX) and the belletristic subset of the CoRoLa corpus. For the DEX dataset, the best results were obtained with the LSTM-based systems at both word (99.32%) and character level (99.43%). For the CoRoLa subset, the CNN-based architecture outperforms at trigram (95.86%) and word level (99.09%) while the LSTM-stacked system obtained the highest accuracy at character level (98.78%). [ABSTRACT FROM AUTHOR]

Land use as contained in geospatial databases constitutes an essential input for different applications such as urban management, regional planning and environmental monitoring. In this paper, a hierarchical deep learning framework is proposed to verify the land use information. For this purpose, a two-step strategy is applied. First, given high-resolution aerial images, the land cover information is determined. To achieve this, an encoder-decoder based convolutional neural network (CNN) is proposed. Second, the pixel-wise land cover information along with the aerial images serves as input for another CNN to classify land use. Because the object catalogue of geospatial databases is frequently constructed in a hierarchical manner, we propose a new CNN-based method aiming to predict land use in multiple levels hierarchically and simultaneously. A so called Joint Optimization (JO) is proposed where predictions are made by selecting the hierarchical tuple over all levels which has the maximum joint class scores, providing consistent results across the different levels. The conducted experiments show that the CNN relying on JO outperforms previous results, achieving an overall accuracy up to 92.5%. In addition to the individual experiments on two test sites, we investigate whether data showing different characteristics can improve the results of land cover and land use classification, when processed together. To do so, we combine the two datasets and undertake some additional experiments. The results show that adding more data helps both land cover and land use classification, especially the identification of underrepresented categories, despite their different characteristics. [ABSTRACT FROM AUTHOR]

Signatures have long been considered to be one of the most accepted and practical means of user verification, despite being vulnerable to skilled forgers. In contrast, EEG signals have more recently been shown to be more difficult to replicate, and to provide better biometric information in response to known a stimulus. In this paper, we propose combining these two biometric traits using a multimodal Siamese Neural Network (mSNN) for improved user verification. The proposed mSNN network learns discriminative temporal and spatial features from the EEG signals using an EEG encoder and from the offline signatures using an image encoder. Features of the two encoders are fused into a common feature space for further processing. A Siamese network then employs a distance metric based on the similarity and dissimilarity of the input features to produce the verification results. The proposed model is evaluated on a dataset of 70 users, comprised of 1400 unique samples. The novel mSNN model achieves a 98.57% classification accuracy with a 99.29% True Positive Rate (TPR) and False Acceptance Rate (FAR) of 2.14%, outperforming the current state-of-the-art by 12.86% (in absolute terms). This proposed network architecture may also be applicable to the fusion of other neurological data sources to build robust biometric verification or diagnostic systems with limited data size. • A multimodal Siamese Neural Network (mSNN) is proposed to combine EEG and Signature data for user verification. • Our mSNN model learns discriminative temporal and spatial features using LSTM and CNN encoders. • Learned features are combined into a common space to build multimodal user verification. • Our mSNN model has a 98.57% classification accuracy — 99.29% True Positive Rate and False Acceptance Rate of 2.14%. • Our mSNN model demonstrates its superiority over traditional and unimodal approaches. [ABSTRACT FROM AUTHOR]