Search

Now and in the future, a face mask is a very important strategy to protect people when a new contagious life threatens disease spread through the air appears. Currently, there is a serious health emergency because of the coronavirus disease 2019 (COVID-19) epidemic. The negative consequences of this pandemic need to be protected in public areas. Numerous methods are advised by the World Health Organization (WHO) to reduce infection rates and prevent depleting the available medical resources in the absence of efficient antivirals. Wearing masks is a non-pharmaceutical strategy to lessen the susceptibility to COVID-19 infection. This research aims to create a face mask identification system that is efficient and uses deep learning, which has proven to be beneficial in many real-world applications. This system has also used a transfer learning method with the MobileNetV2 model to classify people who wear face masks properly, wear face masks improperly, and are without masks. The results demonstrate that the proposed system has an accuracy of 99.4% which is higher than current systems.

Food delivery methods are at the top of the list in today's world. People's attitudes toward food delivery systems are usually influenced by food quality and delivery time. We did a sentiment analysis of consumer comments on the Facebook pages of Food Panda, HungryNaki, Pathao Food, and Shohoz Food, and data was acquired from these four sites’ remarks. In natural language processing (NLP) task, before the model was implemented, we went through a rigorous data pre-processing process that included stages like adding contractions, removing stop words, tokenizing, and more. Four supervised classification techniques are used: extreme gradient boosting (XGB), random forest classifier (RFC), decision tree classifier (DTC), and multi nominal Naive Bayes (MNB). Three deep learning (DL) models are used: convolutional neural network (CNN), long term short memory (LSTM), and recurrent neural network (RNN). The XGB model exceeds all four machine learning (ML) algorithms with an accuracy of 89.64%. LSTM has the highest accuracy rate of the three DL algorithms, with an accuracy of 91.07%. Among ML and DL models, LSTM DL takes the lead to predict the sentiment.

The patients with diseases that cause severe movement disabilities was noticeably increasing. These disabilities made patients unable to carry out their daily activities or interact with their external environment. However, the existence of human-computer interfaces (HCI) gave those patients a new hope to be able to interact once again. HCI enabled these patients to communicate with their environment by recognizing the movement of their eyes. Eye movements are recorded by an electro-oculogram (EOG) through some electrodes that are put vertically and horizontally on the eyes. In this paper, EOG vertical and horizontal signals were analyzed to detect six eye movements (up, down, right, left, double blinking, and center). Three deep learning models namely convolution neural network (CNN), visual geometry group (VGG), and inception had been examined on filtered EOG signals. The experimental results reveal the superiority of the inception model in providing the best average accuracy 96.4%. Accordingly, a writing system is presented based on the detected movements.

In this paper, we have proposed an overcomplete U-Net to perform liver tumor segmentation jointly using a curriculum learning strategy. Liver tumor segmentation is the most prominent and primary step in treating liver cancer and can also help doctors with proper diagnosis and therapy planning. However, it is challenging because of variations in shape, position, and depth of tumors and adjacent boundaries with internal organs around the liver. We have presented a promising solution by designing a U-Net-based segmentation network with two branches: an overcomplete branch to fine grade the small structures and an undercomplete branch to fine grade the high-level structures. This combination allows the network to learn all types of tumor artifacts more accurately. We also changed the conventional learning paradigm to curriculum learning where the input images are fed to the network from easy to hard ones to achieve faster convergence. Finally, our network segments the tumors directly from the whole medical images without the need for segmented liver region of interests (ROIs). The proposed network achieved a DICE score of 75% in tumor segmentation which is a decent value when compared with some existing deep learning methods for liver tumor segmentation.

Congestion on roadways is an issue in many cities, especially at peak times, which causes air and noise pollution and cause pressure on citizens. So, the implementation of intelligent transportation systems (ITSs) is a very important part of smart cities. As a result, the importance of making accurate short-term predictions of traffic flow has significantly increased in recent years. However, the current methods for predicting short-term traffic flow are incapable of effectively capturing the complex non-linearity of traffic flow that affects prediction accuracy. To overcome this problem, this study introduces two novel models. The first model uses two long-short term memory (LSTM) units that can extract the traffic flow temporal features followed by four dense layers to perform the traffic flow prediction. The second model uses two gated recurrent unit (GRU) units that can extract the traffic flow temporal features followed by three dense layers to perform the traffic flow prediction. The two proposed models give promising results on performance measurement system (PEMS), traffic and congestions (TRANCOS) dataset that is firstly used as metadata. So, the two models can do this in specific cases and are able to suddenly capture trend changes.

In recent years, the use of artificial intelligence (AI) for image and video-based crime detection has gained significant attention from law enforcement agencies and security experts. Indeed, deep learning (DL) models can learn complex patterns from data and help law enforcement agencies save time and resources by automatically identifying and tracking potential criminals. This contributes to make deep investigations and better steer their targets’ searches. Among others, handheld firearms and bladed weapons are the most frequent objects encountered at crime scenes. In this paper, we propose a DL-based surveillance system that can detect the presence of tracked objects, such as handheld firearms and bladed weapons, as well as may proceed to alert authorities regarding eventual threats before an incident occurs. After making a comparison of different DL-based object detection techniques, such as you only look once (YOLO), single shot multibox detector (SSD), or faster region-based convolutional neural networks (R-CNN), YOLO achieves the optimal balance of mean average precision (mAP) and inference speed for real-time prediction. Thus, we retain YOLOv5 for the implementation of our solution.

Several industries use clickbait techniques as their strategy to increase the number of readers for their news. Some news companies implement catchy headlines and images in their news article links, with the expectation that the readers will be interested in reading the news and click the provided link. The majority of the news is not hoax news. However, the content might not be as grand as the catchy headlines and images provided to the readers. This research aims to explore the classification model using machine learning to identify if the headlines are classified as clickbait in online news. This research explores several machine learning techniques to classify clickbait in online news and comprehensively explain the results. Several popular machine learning techniques were implemented and explored in this research. The results demonstrate that the model trained with fast large margin provides the best accuracy and classification error (90% and 10%, respectively). Moreover, to improve the performance, bidirectional encoder representations from transformers architecture was used to model clickbait in online news. The best BERT model achieved 98.86% in the test accuracy. BERT model requires more time to train (0.9 hour) compared to machine learning (0.4 hour).

Graphically prescribed patient-specific imaging volumes and local pre-scan volumes are routinely placed by MRI technologists to optimize image quality. However, manual placement of these volumes by MR technologists is time-consuming, tedious, and subject to intra- and inter-operator variability. Resolving these bottlenecks is critical with the rise in abbreviated breast MRI exams for screening purposes. This work proposes an automated approach for the placement of scan and pre-scan volumes for breast MRI. Anatomic 3-plane scout image series and associated scan volumes were retrospectively collected from 333 clinical breast exams acquired on 10 individual MRI scanners. Bilateral pre-scan volumes were also generated and reviewed in consensus by three MR physicists. A deep convolutional neural network was trained to predict both the scan and pre-scan volumes from the 3-plane scout images. The agreement between the network-predicted volumes and the clinical scan volumes or physicist-placed pre-scan volumes was evaluated using the intersection over union, the absolute distance between volume centers, and the difference in volume sizes. The scan volume model achieved a median 3D intersection over union of 0.69. The median error in scan volume location was 2.7 cm and the median size error was 2%. The median 3D intersection over union for the pre-scan placement was 0.68 with no significant difference in mean value between the left and right pre-scan volumes. The median error in the pre-scan volume location was 1.3 cm and the median size error was −2%. The average estimated uncertainty in positioning or volume size for both models ranged from 0.2 to 3.4 cm. Overall, this work demonstrates the feasibility of an automated approach for the placement of scan and pre-scan volumes based on a neural network model.

This paper investigates the impact of several hyperparameters on static malware detection using deep learning, including the number of epochs, batch size, number of layers and neurons, optimisation method, dropout rate, type of activation function, and learning rate. We employed the cAgen tool and grid search optimisation from the scikit-learn Python library to identify the best hyperparameters for our Keras deep learning model. Our experiments reveal that cAgen is more efficient than grid search in finding the optimal parameters, and we find that the selection of hyperparameter values has a significant impact on the model’s accuracy. Specifically, our approach leads to significant improvements in the neural network model’s accuracy for static malware detection on the Ember dataset (from 81.2% to 95.7%) and the Kaggle dataset (from 94% to 98.6%). These results demonstrate the effectiveness of our proposed approach, and have important implications for the field of static malware detection.

Machine Learning (ML) has enabled many image-based object detection and recognition-based solutions in various fields and is the state-of-the-art method for these tasks currently. Therefore, it is of interest to apply this technique to different questions. In this paper, we explore whether it is possible to classify apple cultivars based on fruits using ML methods and images of the apple in question. The goal is to develop a tool that is able to classify the cultivar based on images that could be used in the field. This helps to draw attention to the variety and diversity in fruit growing and to contribute to its preservation. Classifying apple cultivars is a certain challenge in itself, as all apples are similar, while the variety within one class can be high. At the same time, there are potentially thousands of cultivars indicating that the task becomes more challenging when more cultivars are added to the dataset. Therefore, the first question is whether a ML approach can extract enough information to correctly classify the apples. In this paper, we focus on the technical requirements and prerequisites to verify whether ML approaches are able to fulfill this task with a limited number of cultivars as proof of concept. We apply transfer learning on popular image processing convolutional neural networks (CNNs) by retraining them on a custom apple dataset. Afterward, we analyze the classification results as well as possible problems. Our results show that apple cultivars can be classified correctly, but the system design requires some extra considerations.

This journal presents a technique for detecting scratches, cracks and other damage to car bodies using machine learning methods. This method is used to improve process efficiency and checking accuracy and can also reduce the cost and time required for manual inspection. The method includes collecting image datasets of cars in good and damaged condition, followed by preprocessing and segmentation to separate damaged or damaged car parts. not broken. Then, it is followed by a deep learning algorithm, namely You Only Look Once, or Faster Region-based Convolutional Neural Networks, which is used to build a detection model. The model is trained and tuned using the collected data, then evaluated using the test data to measure the accuracy and precision of the detection results. The experimental results show that the proposed method achieves high accuracy and efficiency in detecting scratches, cracks, and other defects on the car body, with precision of an average of more than 70%. This method provides a promising approach to improving the car body inspection process which can be used by taxi companies to help inspect and maintain vehicles more quickly and accurately, to help with insurance, avoid accidents and so on.

With the aid of advancing technologies, the healthcare sector is improving more swiftly than ever, yet there are still many unexplored areas and there is always potential for improvement. Deep Learning (DL), blockchain and the Internet of Medical Things (IoMT), three of the most revolutionary technologies, have aided replace conventional practice employed in the healthcare industry with more sophisticated and efficient procedures. In this research, we propose DON-B-STRESSED (DBS), an innovative framework which integrates DL, blockchain and IoMT technologies to provide a method for the early detection of stress in users donning IoMT devices. Long-term stress, indeed, is well-known to have profound consequences on human wellbeing; hence the need for automated solutions for continual stress monitoring. DON-B-STRESSED stands for DON a Blockchain-based, Secure, Timely, REliable System for StrEss Detection: the premise behind this is that if the above-mentioned technologies can be combined, a system that employs IoMT for users' data collection, DL to properly predict patients' stress levels, and smart contracts deployed on blockchain to ensure secure transactions and health checks, the well-known concerns of harmful effects of long-term stress might be avoided. By using multimodal data from both a chest- and a wrist-worn IoMT device, we have developed a tailored, secure, accurate yet fast real-time system for the early detection of stress signs. The DL part has been evaluated on WESAD, a publicly available dataset for wearable stress and affect detection, reaching an accuracy of more than 99% with LOSO generalization scheme. Furthermore, the scalability and reliability of DON-B-STRESSED are shown by comparison with the findings of past research.

Objective: Deep learning based on convolutional neural networks (CNN) has achieved success in brain-computer interfaces (BCIs) using scalp electroencephalography (EEG). However, the interpretation of the so-called 'black box' method and its application in stereo-electroencephalography (SEEG)-based BCIs remain largely unknown. Therefore, in this paper, an evaluation is performed on the decoding performance of deep learning methods on SEEG signals. Methods: Thirty epilepsy patients were recruited, and a paradigm including five hand and forearm motion types was designed. Six methods, including filter bank common spatial pattern (FBCSP) and five deep learning methods (EEGNet, shallow and deep CNN, ResNet, and a deep CNN variant named STSCNN), were used to classify the SEEG data. Various experiments were conducted to investigate the effect of windowing, model structure, and the decoding process of ResNet and STSCNN. Results: The average classification accuracy for EEGNet, FBCSP, shallow CNN, deep CNN, STSCNN, and ResNet were 35 ± 6.1%, 38 ± 4.9%, 60 ± 3.9%, 60 ± 3.3%, 61 ± 3.2%, and 63 ± 3.1% respectively. Further analysis of the proposed method demonstrated clear separability between different classes in the spectral domain. Conclusion: ResNet and STSCNN achieved the first- and second-highest decoding accuracy, respectively. The STSCNN demonstrated that an extra spatial convolution layer was beneficial, and the decoding process can be partially interpreted from spatial and spectral perspectives. Significance: This study is the first to investigate the performance of deep learning on SEEG signals. In addition, this paper demonstrated that the so-called 'black-box' method can be partially interpreted.

Mid-term load forecasting (MTLF) is of great significance for power system planning, operation, and power trading. However, the mid-term electrical load is affected by the coupling of multiple factors and demonstrates complex characteristics, which leads to low prediction accuracy in MTLF. Furthermore, MTLF is faced with the ``curse of dimensionality'' problem due to a large number of variables. This article proposes an MTLF method based on manifold learning, which can extract the underlying factors of load variations to help improve the accuracy of MTLF and significantly reduce the calculation. Unlike linear dimensionality reduction methods, manifold learning has better nonlinear feature extraction ability and is more suitable for load data with nonlinear characteristics. Furthermore, long short-term memory (LSTM) neural networks are used to establish forecasting models in the low-dimensional space obtained by manifold learning. The proposed MTLF method is tested on independent system operator (ISO) New England datasets, and load forecasting in 24, 168, and 720 h ahead is carried out. The numerical results validate that the proposed method has higher prediction accuracy than many mature methods in the mid-term time scale.

Mobile Edge Computing (MEC) has emerged as a promising paradigm catering to overwhelming explosions of mobile applications, by offloading the compute-intensive tasks to an MEC network for processing. The surging of deep learning brings new vigor and vitality to shape the prospect of intelligent Internet of Things (IoT), and edge intelligence arises to provision real-time deep neural network (DNN) inference services for users. In this paper, we study a novel delay-aware DNN inference throughput maximization problem by accelerating each DNN inference through jointly exploring DNN partitioning and multi-thread parallelism. Specifically, we consider the problem under both offline and online request arrival settings: a set of DNN inference requests is given in advance, and a sequence of DNN inference requests arrives one by one without the knowledge of future arrivals, respectively. We first show that the defined problems are NP-hard. We then devise a novel constant approximation algorithm for the problem under the offline setting. We also propose an online algorithm with a provable competitive ratio for the problem under the online setting. We finally evaluate the performance of the proposed algorithms through experimental simulations. Experimental results demonstrate that the proposed algorithms are promising.

Deep learning is more extensively used in image analysis-based classification of wounds with an aim to facilitate the monitoring of wound prognosis in preventive treatments. In this paper, the classification success of AlexNet architecture in pressure and diabetic foot wound images is discussed. Optimizing training parameters in order to increase the success of Convolutional Neural Network (CNN) architectures is a frequently discussed problem. This paper comparatively examines the effects of optimization of the training parameters of CNN architecture on classification success. The paper examines how the optimizer algorithm, mini-batch size (MBS), maximum epoch number (ME), learning rate (LR), and LearnRateSchedule (LRS) parameters, which are among the training parameters used in combination in architectural training, perform at different values. The best results were obtained with an accuracy of 95.48% at the 10e-4 value of the LR parameter. When the changes in the evaluation metrics during the parameter optimization experiments were examined, it was seen that the LR parameter produced optimum values at 10e-4. As a result, when the Accuracy metric and standard deviations were examined, it was determined only with the LR parameter. No general conclusion could be reached regarding the other parameters.

The distinction between high-grade glioma (HGG) and low-grade glioma (LGG) is generally performed with two-dimensional (2D) image analyses that constitute semi-automated tumor classification. However, a fully automated computer-aided diagnosis (CAD) can only be realized using an adaptive classification framework based on three-dimensional (3D) segmented tumors. In this paper, we handle the classification section of a fully automated CAD related to the aforementioned requirement. For this purpose, a 3D to 2D feature transform strategy (3t2FTS) is presented operating first-order statistics (FOS) in order to form the input data by considering every phase (T1, T2, T1c, and FLAIR) of information on 3D magnetic resonance imaging (3D MRI). Herein, the main aim is the transformation of 3D data analyses into 2D data analyses so as to applicate the information to be fed to the efficient deep learning methods. In other words, 2D identification (2D-ID) of 3D voxels is produced. In our experiments, eight transfer learning models (DenseNet201, InceptionResNetV2, InceptionV3, ResNet50, ResNet101, SqueezeNet, VGG19, and Xception) were evaluated to reveal the appropriate one for the output of 3t2FTS and to design the proposed framework categorizing the 210 HGG–75 LGG instances in the BraTS 2017/2018 challenge dataset. The hyperparameters of the models were examined in a comprehensive manner to reveal the highest performance of the models to be reached. In our trails, two-fold cross-validation was considered as the test method to assess system performance. Consequently, the highest performance was observed with the framework including the 3t2FTS and ResNet50 models by achieving 80% classification accuracy for the 3D-based classification of brain tumors.

One of the ways that celebs maintain their fame in the modern era is by posting updates and photos to social media platforms like Twitter, Instagram, and Facebook. Comments left on their posts, however, expose them to cyberbullying. Cyberbullying, as a form of electronic device-based harassment, negatively impacts the lives of individuals. Thirty famous people from the fields of acting, art, music, politics, sports, and writing were chosen for this research. These notable figures include the top five Twitter followers of Turkey in each demographic. Between December 2019 and December 2020, comment responses for each celebrity were collated. Using the Deep Learning model, we were able to detect abuse content with an accuracy of 89%. Additionally, the percentage of celebrities exposed to cyberbullying by group was presented.

Nowadays, built heritage condition assessment is realized through on-site or photo-aided visual inspections, reporting pathologies manually on drawings, photographs, notes. The knowledge of the state of conservation goes through subjective and time or cost consuming procedures. This is even relevant for a historic building characterized by geometrical and morphological complexity and huge extension, or at risk of collapse. In this context, advancements in the field of Computer Vision and Artificial Intelligence provide an opportunity to address these criticalities. The proposed methodology is based on a Mask R-CNN model, for the detection of decay morphologies on built heritages, and, particularly on historic buildings. The experimentation has been carried out and validated on a highly heterogeneous dataset of images of historic buildings, representative of the regional Architectural Heritage, such as: castles, monasteries, noble buildings, rural buildings. The outcomes highlighted the significance of this remote, non-invasive inspection technique, in support of the technicians in the preliminary knowledge of the building state of conservation, and, most of all, in the decay mapping of some particular classes of alterations (moist area, biological colonization).

Spectral clustering is a well-known clustering algorithm for unsupervised learning, and its improved algorithms have been successfully adapted for many real-world applications. However, traditional spectral clustering algorithms are still facing many challenges to the task of unsupervised learning for large-scale datasets because of the complexity and cost of affinity matrix construction and the eigen-decomposition of the Laplacian matrix. From this perspective, we are looking forward to finding a more efficient and effective way by adaptive neighbor assignments for affinity matrix construction to address the above limitation of spectral clustering. It tries to learn an affinity matrix from the view of global data distribution. Meanwhile, we propose a deep learning framework with fully connected layers to learn a mapping function for the purpose of replacing the traditional eigen-decomposition of the Laplacian matrix. Extensive experimental results have illustrated the competitiveness of the proposed algorithm. It is significantly superior to the existing clustering algorithms in the experiments of both toy datasets and real-world datasets.

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link. Semi-supervised deep learning (SSDL) is a popular strategy to leverage unlabelled data for machine learning when labelled data is not readily available. In real-world scenarios, different unlabelled data sources are usually available, with varying degrees of distribution mismatch regarding the labelled datasets. It begs the question which unlabelled dataset to choose for good SSDL outcomes. ftentimes, semantic heuristics are used to match unlabelled data with labelled data. However, a quantitative and systematic approach to this election problem would be preferable. In this work, we first test the SSDL MixMatch algorithm under various distribution mismatch configurations to study the impact on SSDL accuracy. Then, we propose a quantitative unlabelled dataset selection heuristic based on dataset dissimilarity measures. These are designed to systematically assess how distribution mismatch between the labelled and unlabelled datasets affects MixMatch performance. We refer to our proposed method as deep dataset dissimilarity measures (DeDiMs), designed to compare labelled and unlabelled datasets. They use the feature space of a generic Wide-ResNet, can be applied prior to learning, are quick to evaluate and model agnostic. The strong correlation in our tests between MixMatch accuracy and the proposed DeDiMs suggests that this approach can be a good fit for quantitatively ranking different unlabelled datasets prior to SSDL training.

This paper introduces an algorithm for Facial Expression Recognition (FER) using deep Siamese Neural Networks (SNNs) that preserve the local structure of images in the embedding similarity space. We designed the network to reveal the input pairs similarity by comparing features through a designed metric. Furthermore, we developed a novel image pairing (i.e., positive and negative pairs) strategy technique to train our Siamese model. Our Siamese model comprises of a verification framework and an identification framework to learn a joint embedding space. The verification path reduces the intra-class variations by minimizing the distance between the extracted features from the same class, while the identification path increases the inter-class variations by maximizing the distance between the features extracted from different classes. We apply transfer learning to only use the identification model for facial expression classification. We evaluated our algorithm using AffectNet, FER2013, and Compound Facial Expressions of Emotion (CFEE) datasets, where better results are achieved compared to other deep learning-based approaches.

International audience; Multispectral snapshot cameras fitted with a multispectral filter array (MSFA) acquire several spectral bands in one shot and provide a raw mosaic image in which a single channel value is available at each pixel. Texture features are classically extracted from fully-defined images that are estimated by demosaicing. Such an estimation may however cause spatio-spectral artifacts. Moreover, texture feature extraction becomes computationally inefficient and yields to high-dimensional features as the number of bands increases. In this paper, we propose an original approach based on a convolutional neural network called MSFA-Net to capture spatio-spectral interactions in raw images at reduced computation costs. Experiments of multispectral image classification and outdoor image segmentation show that the proposed approach outperforms several hand-crafted and deep learning-based feature extractors.

COVID-19 has caused disruptions to many aspects of everyday life. To reduce the impact of this pandemic, its spreading must be controlled via face mask wearing. Manually mask-checking for everybody is embarrassing and uncontrollable. Hence, the proposed technique is used to help for automatic mask-checking based on deep learning platforms with real-time surveillance live infra-red (IR) camera. In this paper, two recent object detection platforms, named, you only look once version 3 (YOLOv3) and TensorFlow lite are adopted to accomplish this task. The two models are trained with a dataset consisting of images of persons with/without masks. This work is simulated with Google Colab then tested in real-time on an embedded device mated with fast GPU called Raspberry Pi 4 model B, 8 GB RAM. A comparison is made between the two models to verify their performance in relation to their precision rate and processing time. The work of this paper is also succeeded to realize multiple face masks real-time detection up to 10 facemasks in a single scene with high inference speed. Temperature is also measured using IR touchless sensor for each person with sound alarming to alert fever. The presented detector is cheap, light, small, and fast, with 99% accuracy rate during training and testing.

Generative adversarial networks (GANs) are a hot research topic recently. GANs have been widely studied since 2014, and a large number of algorithms have been proposed. However, there is few comprehensive study explaining the connections among different GANs variants, and how they have evolved. In this paper, we attempt to provide a review on various GANs methods from the perspectives of algorithms, theory, and applications. Firstly, the motivations, mathematical representations, and structure of most GANs algorithms are introduced in details. Furthermore, GANs have been combined with other machine learning algorithms for specific applications, such as semi-supervised learning, transfer learning, and reinforcement learning. This paper compares the commonalities and differences of these GANs methods. Secondly, theoretical issues related to GANs are investigated. Thirdly, typical applications of GANs in image processing and computer vision, natural language processing, music, speech and audio, medical field, and data science are illustrated. Finally, the future open research problems for GANs are pointed out.

Facial micro-expression (ME) can disclose genuine and concealed human feelings. It makes MEs extensively useful in real-world applications pertaining to affective computing and psychology. Unfortunately, they are induced by subtle facial movements for a short duration of time, which makes the ME recognition, a highly challenging problem even for human beings. In automatic ME recognition, the well-known features encode either incomplete or redundant information, and there is a lack of sufficient training data. The proposed method, Micro-Expression Recognition by Analysing Spatial and Temporal Characteristics, MERASTC mitigates these issues for improving the ME recognition. It compactly encodes the subtle deformations using action units (AUs), landmarks, gaze, and appearance features of all the video frames while preserving most of the relevant ME information. Furthermore, it improves the efficacy by introducing a novel neutral face normalization for ME and initiating the utilization of gaze features in deep learning based ME recognition. The features are provided to the 2D convolutional neural network that jointly analyses the spatial and temporal behavior for correct ME classification. Experimental results on the publicly available datasets indicate that the proposed method exhibits better performance than the well-known methods.

With the rapid development of the Internet, readers tend to share their views and emotions about news events. Predicting these emotions provides a vital role in social media applications (e.g., sentiment retrieval, opinion summary, and election prediction). However, news articles usually consist of objective texts that lack emotion words, making emotion prediction challenging. From prior studies, we know that comments that come directly from readers are full of emotions. Therefore, in this article, we propose a deep learning framework that first merges article and comment information to predict readers' emotions. At the same time, in the prediction process, we design a pseudo comment representation for unpublished news articles by the comments of published news. In addition, a better model is required to encode articles that contain implicit emotions. To solve this problem, we propose a block emotion attention network (BEAN) to encode news articles better. It includes an emotion attention mechanism and a hierarchical structure to capture emotion words and generate structural information during encoding. Experiments performed on three public datasets show that BEAN achieves the state-of-the-art average Pearson (AP) and accuracy (Acc@1). Moreover, results on four self-collected datasets show that both the introduction of emotional comments and BEAN in our framework improve the ability to predict readers' emotions.

Deep learning-enabled smartphone-based image processing has significant advantages in the development of point-of-care diagnostics. Conventionally, most deep-learning applications require task specific large scale expertly annotated datasets. Therefore, these algorithms are oftentimes limited only to applications that have large retrospective datasets available for network development. Here, we report the possibility of utilizing adversarial neural networks to overcome this challenge by expanding the utility of non-specific data for the development of deep learning models. As a clinical model, we report the detection of fentanyl, a small molecular weight drug that is a type of opioid, at the point-of-care using a deep-learning empowered smartphone assay. We used the catalytic property of platinum nanoparticles (PtNPs) in a smartphone-enabled microchip bubbling assay to achieve high analytical sensitivity (detecting fentanyl at concentrations as low as 0.23 ng mL

The present study investigates the effectiveness of a deep learning neural network for non-invasively localizing the seizure onset zone (SOZ) using multi-modal MRI data that are clinically acquired from children with drug-resistant epilepsy. A cortical parcellation was applied to localize the SOZ in cortical nodes of the epileptogenic hemisphere. At each node, the laminar surface analysis was followed to sample 1) the relative intensity of gray matter and white matter in multi-modal MRI and 2) the neighboring white matter connectivity using diffusion tractography edge strengths. A cross-validation was employed to train and test all layers of a multi-scale residual neural network (msResNet) that can classify SOZ node in an end-to-end fashion. A prediction probability of a given node belonging to the SOZ class was proposed as a non-invasive MRI marker of seizure onset likelihood. In an independent validation cohort, the proposed MRI marker provided a very large effect size of Cohen's d = 1.21 between SOZ and non-SOZ, and classified SOZ with a balanced accuracy of 0.75 in lesional and 0.67 in non-lesional MRI groups. The subsequent multi-variate logistic regression found the incorporation of the proposed MRI marker into interictal intracranial EEG (iEEG) markers further improves the differentiation between the epileptogenic focus (defined as SOZ resected during surgery) and non-epileptogenic sites (i.e., non-SOZ sites preserved during surgery) up to 15 % in non-lesional MRI group, suggesting that the proposed MRI marker could improve the localization of epileptogenic foci for successful pediatric epilepsy surgery.

This study introduces a deep learning approach to accurately predict challenging mechanical environments that possibly cause decreasing postural stability.Dual-axis robotic platforms were utilized to simulate various environments and collect center-of-pressure data during narrow and wide stance. A convolutional neural network (CNN) was developed to predict environmental conditions given segmented time-series balance data. Different window sizes were examined to investigate its minimal length for reliable prediction. Effectiveness of the presented CNN was additionally compared with that of conventional machine learning models. Its applicability with low sampled data or more natural stance data was then evaluated.The CNN achieved above 94.5% in the overall prediction accuracy even with 2.5-second length postural sway data, which cannot be achieved by traditional machine learning (ps0.05). Increasing data length beyond 2.5 seconds slightly improved the accuracy of CNN but substantially increased training time (60% longer). Importantly, results from averaged normalized confusion matrices revealed that CNN is much more capable of differentiating the mid-level environmental condition. Deep learning could also produce comparable performance in predicting environments even with much lower sampled data or with standing posture changed.CNN removed the burden of feature preparation and accurately predicted environments when dealing with short-length data. It also indicated potentials to real life applications.This study contributes to the advancement of wearable devices and human interactive robots (e.g., exoskeletons and prostheses) by predicting environmental contexts and preventing potential falls.

Humans’ fundamental need is interaction with each other such as using conversation or speech. Therefore, it is crucial to analyze speech using computer technology to determine emotions. The speech emotion recognition (SER) method detects emotions in speech by examining various aspects. SER is a supervised method to decide the emotion class in speech. This research proposed a multimodal SER model using one of the deep learning based enhancement techniques, which is the attention mechanism. Additionally, this research addresses the imbalanced dataset problem in the SER field using generative adversarial networks (GAN) as a data augmentation technique. The proposed model achieved an excellent evaluation performance of 0.96 or 96% for the proposed GAN configuration. This work showed that the GAN method in the multimodal SER model could enhance performance and create a balanced dataset.

Recently, Panama wilt disease that attacks banana leaves has caused enormous economic losses to farmers. Early detection of this disease and necessary preventive measures can avoid economic damage. This paper proposes an improved method to predict Panama wilt disease based on symptoms using an agro deep learning algorithm. The proposed deep learning model for detecting Panama wilts disease is essential because it can help accurately identify infected plants in a timely manner. It can be instrumental in large-scale agricultural operations where Panama wilts disease could spread quickly and cause significant crop loss. Additionally, deep learning models can be used to monitor the effectiveness of treatments and help farmers make informed decisions about how to manage the disease best. This method is designed to predict the severity of the disease and its consequences based on the arrangement of color and shape changes in banana leaves. The present proposed method is compared with its previous methods, and it achieved 91.56% accuracy, 91.61% precision, 88.56% recall and 81.56% F1-score.

In order to meet the world's demand for food production, farmers and producers have improved and increased their agricultural production capabilities, leading to a profit acceleration in the field. However, this growth has also caused significant environmental damage due to the widespread use of herbicides. Weeds competing with crops result in lower crop yields and a 30% increase in losses. To rationalize the use of these herbicides, it would be more effective to detect the presence of weeds before application, allowing for the selection of the appropriate herbicide and application only in areas where weeds are present. The focus of this paper is to define a pipeline for detecting weeds in images through the use of a Mask R-CNN-based weed classification and segmentation module. The model was initially trained locally on our machine, but limitations and issues with training time prompted the team to switch to cloud solutions for training.

Aims This study aims to evaluate the ability of a deep-learning-based cardiovascular disease (CVD) retinal biomarker, Reti-CVD, to identify individuals with intermediate- and high-risk for CVD. Methods and results We defined the intermediate- and high-risk groups according to Pooled Cohort Equation (PCE), QRISK3, and modified Framingham Risk Score (FRS). Reti-CVD’s prediction was compared to the number of individuals identified as intermediate- and high-risk according to standard CVD risk assessment tools, and sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated to assess the results. In the UK Biobank, among 48 260 participants, 20 643 (42.8%) and 7192 (14.9%) were classified into the intermediate- and high-risk groups according to PCE, and QRISK3, respectively. In the Singapore Epidemiology of Eye Diseases study, among 6810 participants, 3799 (55.8%) were classified as intermediate- and high-risk group according to modified FRS. Reti-CVD identified PCE-based intermediate- and high-risk groups with a sensitivity, specificity, PPV, and NPV of 82.7%, 87.6%, 86.5%, and 84.0%, respectively. Reti-CVD identified QRISK3-based intermediate- and high-risk groups with a sensitivity, specificity, PPV, and NPV of 82.6%, 85.5%, 49.9%, and 96.6%, respectively. Reti-CVD identified intermediate- and high-risk groups according to the modified FRS with a sensitivity, specificity, PPV, and NPV of 82.1%, 80.6%, 76.4%, and 85.5%, respectively. Conclusion The retinal photograph biomarker (Reti-CVD) was able to identify individuals with intermediate and high-risk for CVD, in accordance with existing risk assessment tools.

There are various challenges in identifying the speakers accurately. The Extraction of discriminative features is a vital task for accurate identification in the speaker identification task. Nowadays, speaker identification is widely investigated using deep learning. The complex and noisy speech data affects the performance of Mel Frequency Cepstral Coefficients (MFCC); hence, MFCC fails to represent the speaker characteristics accurately. In this proposed work, a novel text-independent speaker identification system is developed to enhance the performance by fusion of Log-MelSpectrum and excitation features. The excitation information is obtained due to the vibration of vocal folds, and it is represented using Linear Prediction (LP) residual. The various types of features extracted from the excitation are residual phase, sharpness, Energy of Excitation (EoE), and Strength of Excitation (SoE). The extracted features were processed with the dilated convolution neural network (dilated CNN) to fulfill the identification task. The extensive evaluation showed that the fusion of excitation features gives better results than the existing methods. The accuracy reaches 94.12% for 11 complex classes and 91.34% for 80 speakers, and Equal Error Rate (EER) is reduced to 1.16% for the proposed model. The proposed model is tested with the Librispeech corpus using Matlab 2021b tool, outperforming the existing baseline models. The proposed model achieves an accuracy improvement of 1.34% compared to the baseline system.

Gait analysis is a powerful technique that detects and identifies foot disorders and walking irregularities, including pronation, supination, and unstable foot movements. Early detection can help prevent injuries, correct walking posture, and avoid the need for surgery or cortisone injections. Traditional gait analysis methods are expensive and only available in laboratory settings, but new wearable technologies such as AI and IoT-based devices, smart shoes, and insoles have the potential to make gait analysis more accessible, especially for people who cannot easily access specialized facilities. This research proposes a novel approach using IoT, edge computing, and tiny machine learning (TinyML) to predict gait patterns using a microcontroller-based device worn on a shoe. The device uses an inertial measurement unit (IMU) sensor and a TinyML model on an advanced RISC machines (ARM) chip to classify and predict abnormal gait patterns, providing a more accessible, cost-effective, and portable way to conduct gait analysis.

In this study, a basic insect detection system consisting of a manual-focus camera, a Jetson Nano—a low-cost, low-power single-board computer, and a trained deep learning model was developed. The model was validated through a live visual feed. Detecting, classifying, and monitoring insect pests in a grain storage or food facility in real time is vital to making insect control decisions. The camera captures the image of the insect and passes it to a Jetson Nano for processing. The Jetson Nano runs a trained deep-learning model to detect the presence and species of insects. With three different lighting situations: white LED light, yellow LED light, and no lighting condition, the detection results are displayed on a monitor. Validating using F1 scores and comparing the accuracy based on light sources, the system was tested with a variety of stored grain insect pests and was able to detect and classify adult cigarette beetles and warehouse beetles with acceptable accuracy. The results demonstrate that the system is an effective and affordable automated solution to insect detection. Such an automated insect detection system can help reduce pest control costs and save producers time and energy while safeguarding the quality of stored products.

Facial expressions constitute one of the most effective and instinctive methods that allow people to communicate their emotions and intentions. In this context, the both Machine Learning (ML) and Convolutional Neural Networks (CNNs) have been used for emotion recognition. Efficient recognition systems are required for good human-computer interaction. However, facial expression recognition is related to several methods that impact the performance of facial recognition systems. In this paper, we demonstrate a state-of-the-art of 65% accuracy on the FER2013 dataset by leveraging numerous techniques from recent research and we also proposed some new methods for improving accuracy by combining CNN architectures such as VGG-16 and Resnet-50 with auxiliary datasets such as JAFFE and CK. To predict emotions, we used a second approach based on geometric features and facial landmarks to calculate and transmit the feature vector to the SVM model. The results show that the ResNet50 model outperforms all other emotion prediction models in real time by maximizing.

Cognitive disorders are prevalent in people with HIV (PWH) despite antiretroviral therapy. Given the heterogeneity of cognitive disorders in PWH in the current era and evidence that these disorders have different etiologies and risk factors, scientific rationale is growing for using data-driven models to identify biologically defined subtypes (biotypes) of these disorders. Here, we discuss the state of science using machine learning to understand cognitive phenotypes in PWH and their associated comorbidities, biological mechanisms, and risk factors. We also discuss methods, example applications, challenges, and what will be required from the field to successfully incorporate machine learning in research on cognitive disorders in PWH. These topics were discussed at the National Institute of Mental Health meeting on “Biotypes of CNS Complications in People Living with HIV” held in October 2021. These ongoing research initiatives seek to explain the heterogeneity of cognitive phenotypes in PWH and their associated biological mechanisms to facilitate clinical management and tailored interventions.

Data augmentation is an important procedure in deep learning. GAN-based data augmentation can be utilized in many domains. For instance, in the credit card fraud domain, the imbalanced dataset problem is a major one as the number of credit card fraud cases is in the minority compared to legal payments. On the other hand, generative techniques are considered effective ways to rebalance the imbalanced class issue, as these techniques balance both minority and majority classes before the training. In a more recent period, Generative Adversarial Networks (GANs) are considered one of the most popular data generative techniques as they are used in big data settings. This research aims to present a survey on data augmentation using various GAN variants in the credit card fraud detection domain. In this survey, we offer a comprehensive summary of several peer-reviewed research papers on GAN synthetic generation techniques for fraud detection in the financial sector. In addition, this survey includes various solutions proposed by different researchers to balance imbalanced classes. In the end, this work concludes by pointing out the limitations of the most recent research articles and future research issues, and proposes solutions to address these problems.

Since it can be used to manage and estimate oil reserves, the inventory of oil tanks is essential for both the economy and the military applications. Considering oil tanks contain valuable materials required for transportation and industrial production, they are a significant type of target. Oil tank detection techniques have several uses, including monitoring disasters, preventing oil leaks, designing cities, and assessing damage. Huge amount of satellite imagery has recently been available and it is used in both the military and civil applications. The new spaceborne sensors' higher resolution enables the detection of targeted objects. Therefore, remote sensing instruments provide ideal tools for oil tank detection task. Conventional approaches for oil tank detection from high resolution remote sensing imagery generally relies on geometric shape, structure, contract differences and color information of the boundary or hand-crafted features. However, these methods come along with vulnerabilities and hence it can be challenging to obtain accurate detection in the presence of a number of disturbance elements, particularly a wide range of colours, size variations, and the shadows that view angle and illumination create. Therefore, deep learning-based methods can provide a big advantage for solution of this task. In this regard, this study employs four YOLO models namely YOLOv5, YOLOX, YOLOv6 and YOLOv7 for oil tank detection from high-resolution optical imagery. Our results show that YOLOv7 and YOLOv5 architectures provide more accurate detections with mean average precision values of 68.11% and 69.69%, respectively. The experiments and visual inspections reveal efficiency, generalization and transferability of these models.

Deep learning models have revolutionized research fields like computer vision and natural language processing by outperforming traditional models in multiple tasks. However, the field of time series analysis, especially time series forecasting, has not seen a similar revolution, despite forecasting being one of the most prominent tasks of predictive data analytics. One crucial problem for time series forecasting is the lack of large, domain-independent benchmark datasets and a competitive research environment, e.g., annual large-scale challenges, that would spur the development of new models, as was the case for CV and NLP. Furthermore, the focus of time series forecasting research is primarily domain-driven, resulting in many highly individual and domain-specific datasets. Consequently, the progress in the entire field is slowed down due to a lack of comparability across models trained on a single benchmark dataset and on a variety of different forecasting challenges. In this paper, we first explore this problem in more detail and derive the need for a comprehensive, domain-unspecific overview of the state-of-the-art of commonly used datasets for prediction tasks. In doing so, we provide an overview of these datasets and improve comparability in time series forecasting by introducing a method to find similar datasets which can be utilized to test a newly developed model. Ultimately, our survey paves the way towards developing a single widely used and accepted benchmark dataset for time series data, built on the various frequently used datasets surveyed in this paper.

Considerable progress has recently been made in natural language processing: deep learning algorithms are increasingly able to generate, summarize, translate and classify texts. Yet, these language models still fail to match the language abilities of humans. Predictive coding theory offers a tentative explanation to this discrepancy: while language models are optimized to predict nearby words, the human brain would continuously predict a hierarchy of representations that spans multiple timescales. To test this hypothesis, we analysed the functional magnetic resonance imaging brain signals of 304 participants listening to short stories. First, we confirmed that the activations of modern language models linearly map onto the brain responses to speech. Second, we showed that enhancing these algorithms with predictions that span multiple timescales improves this brain mapping. Finally, we showed that these predictions are organized hierarchically: frontoparietal cortices predict higher-level, longer-range and more contextual representations than temporal cortices. Overall, these results strengthen the role of hierarchical predictive coding in language processing and illustrate how the synergy between neuroscience and artificial intelligence can unravel the computational bases of human cognition.

Weeds are mainly spread by weed seeds being mixed with agricultural and forestry crop seeds, grain, animal hair, and other plant products, and disturb the growing environment of target plants such as crops and wild native plants. The accurate and efficient classification of weed seeds is important for the effective management and control of weeds. However, classification remains mainly dependent on destructive sampling-based manual inspection, which has a high cost and rather low flux. We considered that this problem could be solved using a nondestructive intelligent image recognition method. First, on the basis of the establishment of the image acquisition system for weed seeds, images of single weed seeds were rapidly and completely segmented, and a total of 47 696 samples of 140 species of weed seeds and foreign materials remained. Then, six popular and novel deep Convolutional Neural Network (CNN) models are compared to identify the best method for intelligently identifying 140 species of weed seeds. Of these samples, 33 600 samples are randomly selected as the training dataset for model training, and the remaining 14 096 samples are used as the testing dataset for model testing. AlexNet and GoogLeNet emerged from the quantitative evaluation as the best methods. AlexNet has strong classification accuracy and efficiency (low time consumption), and GoogLeNet has the best classification accuracy. A suitable CNN model for weed seed classification could be selected according to specific identification accuracy requirements and time costs of applications. This research is beneficial for developing a detection system for weed seeds in various applications. The resolution of taxonomic issues and problems associated with the identification of these weed seeds may allow for more effective management and control.

Leaf disease recognition using image processing and deep learning techniques is currently a vibrant research area. Most studies have focused on recognizing diseases from images of whole leaves. This approach limits the resulting models’ ability to estimate leaf disease severity or identify multiple anomalies occurring on the same leaf. Recent studies have demonstrated that classifying leaf diseases based on individual lesions greatly enhances disease recognition accuracy. In those studies, however, the lesions were laboriously cropped by hand. This study proposes a semi-automatic algorithm that facilitates the fast and efficient preparation of datasets of individual lesions and leaf image pixel maps to overcome this problem. These datasets were then used to train and test lesion classifier and semantic segmentation Convolutional Neural Network (CNN) models, respectively. We report that GoogLeNet’s disease recognition accuracy improved by more than 15% when diseases were recognized from lesion images compared to when disease recognition was done using images of whole leaves. A CNN model which performs semantic segmentation of both the leaf and lesions in one pass is also proposed in this paper. The proposed KijaniNet model achieved state-of-the-art segmentation performance in terms of mean Intersection over Union (mIoU) score of 0.8448 and 0.6257 for the leaf and lesion pixel classes, respectively. In terms of mean boundary F1 score, the KijaniNet model attained 0.8241 and 0.7855 for the two pixel classes, respectively. Lastly, a fully automatic algorithm for leaf disease recognition from individual lesions is proposed. The algorithm employs the semantic segmentation network cascaded to a GoogLeNet classifier for lesion-wise disease recognition. The proposed fully automatic algorithm outperforms competing methods in terms of its superior segmentation and classification performance despite being trained on a small dataset.

Deep learning techniques have proven effective in many applications, but these implementations mostly apply to data in one or two dimensions. Handling 3D data is more challenging due to its irregularity and complexity, and there is a growing interest in adapting deep learning techniques to the 3D domain. A recent successful approach called MeshCNN consists of a set of convolutional and pooling operators applied to the edges of triangular meshes. While this approach produced superb results in classification and segmentation of 3D shapes, it can only be applied to edges of a mesh, which can constitute a disadvantage for applications where the focuses are other primitives of the mesh. In this study, we propose face-based and vertex-based operators for mesh convolutional networks. We design two novel architectures based on the MeshCNN network that can operate on faces and vertices of a mesh, respectively. We demonstrate that the proposed face-based architecture outperforms the original MeshCNN implementation in mesh classification and mesh segmentation, setting the new state of the art on benchmark datasets. In addition, we extend the vertex-based operator to fit in the Point2Mesh model for mesh reconstruction from clean, noisy, and incomplete point clouds. While no statistically significant performance improvements are observed, the model training and inference time are reduced by the proposed approach by 91% and 20%, respectively, as compared with the original Point2Mesh model.