Search

Personality determines various human daily and working behaviours. Recently, a large number of automatic personality computing approaches have been developed to predict either the apparent or self-reported personality of the subject based on non-verbal audio-visual behaviours. However, most of them suffer from complex and dataset-specific pre-processing steps and model training tricks. In the absence of a standardized benchmark with consistent experimental settings, it is not only impossible to fairly compare the real performances of these personality computing models but also makes them difficult to be reproduced. This paper presents the first reproducible audio-visual benchmark to provide a fair and consistent evaluation of eight existing personality computing models (e.g., audio, visual and audio-visual) and seven standard deep learning models on both self-reported and apparent personality recognition tasks. Building upon a set of benchmarked models, we also investigate the impact of two previously-used long-term modelling strategies for summarising short-term/frame-level predictions on personality computing results. We comprehensively investigate all benchmarked models on two publicly available datasets, ChaLearn First Impression and UDIVA self-reported personality datasets, and conclude: (i) apparent personality traits, inferred from facial behaviours by most benchmarked deep learning models, show more reliability than self-reported ones; (ii) visual models frequently achieved superior performances than audio models on personality recognition; (iii) non-verbal behaviours contribute differently in predicting different personality traits; and (iv) our reproduced personality computing models generally achieved worse performances than their original reported results. [ABSTRACT FROM AUTHOR]

Heart sound auscultation has been applied in clinical usage for early screening of cardiovascular diseases. Due to the high demand for auscultation expertise, automatic auscultation can help with auxiliary diagnosis and reduce the burden of training professional clinicians. Nevertheless, there is a limit to classic machine learning's performance improvement in the era of Big Data. Deep learning has outperformed classic machine learning in many research fields, as it employs more complex model architectures with a stronger capability of extracting effective representations. Moreover, it has been successfully applied to heart sound analysis in the past years. As most review works about heart sound analysis were carried out before 2017, the present survey is the first to work on a comprehensive overview to summarise papers on heart sound analysis with deep learning published in 2017–2022. This work introduces both classic machine learning and deep learning for comparison, and further offer insights about the advances and future research directions in deep learning for heart sound analysis. [ABSTRACT FROM AUTHOR]

Detecting wafer map anomalies is crucial for preventing yield loss in semiconductor fabrication, although intricate patterns and resource-intensive labeled data prerequisites hinder precise deep-learning segmentation. This paper presents an innovative, unsupervised method for detecting pixel-level anomalies in wafer maps. It utilizes an efficient dual attention module with a knowledge distillation network to learn defect distributions without anomalies. Knowledge transfer is achieved by distilling information from a pre-trained teacher into a student network with similar architecture, except an efficient dual attention module is incorporated atop the teacher network’s feature pyramid hierarchies, which enhances feature representation and segmentation across pyramid hierarchies that selectively emphasize relevant and discard irrelevant features by capturing contextual associations in positional and channel dimensions. Furthermore, it enables student networks to acquire an improved knowledge of hierarchical features to identify anomalies across different scales accurately. The dissimilarity in feature pyramids acts as a discriminatory function, predicting the likelihood of an abnormality, resulting in highly accurate pixel-level anomaly detection. Consequently, our proposed method excelled on the WM-811K and MixedWM38 datasets, achieving AUROC, AUPR, AUPRO, and F1-Scores of (99.65%, 99.35%), (97.31%, 92.13%), (90.76%, 84.66%) respectively, alongside an inference speed of 3.204 FPS, showcasing its high precision and efficiency. [ABSTRACT FROM AUTHOR]

Multi-modal classifiers for emotion recognition have become prominent, as the emotional states of subjects can be more comprehensively inferred from Electroencephalogram (EEG) signals and eye movements. However, existing classifiers experience a decrease in performance due to the distribution shift when applied to new users. Unsupervised domain adaptation (UDA) emerges as a solution to address the distribution shift between subjects by learning a shared latent feature space. Nevertheless, most UDA approaches focus on a single modality, while existing multi-modal approaches do not consider that fine-grained structures should also be explicitly aligned and the learned feature space must be discriminative. In this paper, we propose Coarse and Fine-grained Distribution Alignment with Correlated and Separable Features (CFDA-CSF), which performs a coarse alignment over the global feature space, and a fine-grained alignment between modalities from each domain distribution. At the same time, the model learns intra-domain correlated features, while a separable feature space is encouraged on new subjects. We conduct an extensive experimental study across the available sessions on three public datasets for multi-modal emotion recognition: SEED, SEED-IV, and SEED-V. Our proposal effectively improves the recognition performance in every session, achieving an average accuracy of 93.05%, 85.87% and 91.20% for SEED; 85.72%, 89.60%, and 86.88% for SEED-IV; and 88.49%, 91.37% and 91.57% for SEED-V. [ABSTRACT FROM AUTHOR]

Recognizing human emotions from complex, multivariate, and non-stationary electroencephalography (EEG) time series is essential in affective brain-computer interface. However, because continuous labeling of ever-changing emotional states is not feasible in practice, existing methods can only assign a fixed label to all EEG timepoints in a continuous emotion-evoking trial, which overlooks the highly dynamic emotional states and highly non-stationary EEG signals. To solve the problems of high reliance on fixed labels and ignorance of time-changing information, in this paper we propose a time-aware sampling network (TAS-Net) using deep reinforcement learning (DRL) for unsupervised emotion recognition, which is able to detect key emotion fragments and disregard irrelevant and misleading parts. Specifically, we formulate the process of mining key emotion fragments from EEG time series as a Markov decision process and train a time-aware agent through DRL without label information. First, the time-aware agent takes deep features from a feature extractor as input and generates sample-wise importance scores reflecting the emotion-related information each sample contains. Then, based on the obtained sample-wise importance scores, our method preserves top-X continuous EEG fragments with relevant emotion and discards the rest. Finally, we treat these continuous fragments as key emotion fragments and feed them into a hypergraph decoding model for unsupervised clustering. Extensive experiments are conducted on three public datasets (SEED, DEAP, and MAHNOB-HCI) for emotion recognition using leave-one-subject-out cross-validation, and the results demonstrate the superiority of the proposed method against previous unsupervised emotion recognition methods. The proposed TAS-Net has great potential in achieving a more practical and accurate affective brain-computer interface in a dynamic and label-free circumstance. [ABSTRACT FROM AUTHOR]

With the development of digital medical technology, ubiquitous smartphones are emerging as valuable tools for the detection of complex and elusive diseases. This paper exploits smartphone walking recording for early detection of Parkinson's disease (PD) and finds that walking recording empowered by deep learning is a valid digital biomarker for early-recognizing PD patients. Specifically, the inertial sensor data is preprocessed, including normalization, scaling, and rotation, and then the processed data is fed into the proposed deep NeuroEnhanceNet. Finally, determine the individual prediction score using the PD-prone strategy and generate the detection results. The proposed deep NeuroEnhanceNet, specifically designed for inertial sensor data, can focus on both the long-term data characteristics within a single channel and the inter-channel correlations. Our method obtains a low false negative rate of 0.053 for the early detection of PD. We further analyze and compare the effectiveness of digital biomarkers captured from the walking and resting processes for early detection of PD. All the code for this work is available at: https://github.com/heyiyia/NeuroEnhanceNet.

Training an accurate classifier for EEG-based brain-computer interface (BCI) requires EEG data from a large number of users, whereas protecting their data privacy is a critical consideration. Federated learning (FL) is a promising solution to this challenge. This paper proposes Federated classification with local Batch-specific batch normalization and Sharpness-aware minimization (FedBS) for privacy protection in EEG-based motor imagery (MI) classification. FedBS utilizes local batch-specific batch normalization to reduce data discrepancies among different clients, and sharpness-aware minimization optimizer in local training to improve model generalization. Experiments on three public MI datasets using three popular deep learning models demonstrated that FedBS outperformed six state-of-the-art FL approaches. Remarkably, it also outperformed centralized training, which does not consider privacy protection at all. In summary, FedBS protects user EEG data privacy, enabling multiple BCI users to participate in large-scale machine learning model training, which in turn improves the BCI decoding accuracy.

Sleep staging serves as a fundamental assessment for sleep quality measurement and sleep disorder diagnosis. Although current deep learning approaches have successfully integrated multimodal sleep signals, enhancing the accuracy of automatic sleep staging, certain challenges remain, as follows: 1) optimizing the utilization of multi-modal information complementarity, 2) effectively extracting both long- and short-range temporal features of sleep information, and 3) addressing the class imbalance problem in sleep data. To address these challenges, this paper proposes a two-stream encode-decoder network, named TSEDSleepNet, which is inspired by the depth sensitive attention and automatic multi-modal fusion (DSA2F) framework. In TSEDSleepNet, a two-stream encoder is used to extract the multiscale features of electrooculogram (EOG) and electroencephalogram (EEG) signals. And a self-attention mechanism is utilized to fuse the multiscale features, generating multi-modal saliency features. Subsequently, the coarser-scale construction module (CSCM) is adopted to extract and construct multi-resolution features from the multiscale features and the salient features. Thereafter, a Transformer module is applied to capture both long- and short-range temporal features from the multi-resolution features. Finally, the long- and short-range temporal features are restored with low-layer details and mapped to the predicted classification results. Additionally, the Lovász loss function is applied to alleviate the class imbalance problem in sleep datasets. Our proposed method was tested on the Sleep-EDF-39 and Sleep-EDF-153 datasets, and it achieved classification accuracies of 88.9% and 85.2% and Macro-F1 scores of 84.8% and 79.7%, respectively, thus outperforming conventional traditional baseline models. These results highlight the efficacy of the proposed method in fusing multi-modal information. This method has potential for application as an adjunct tool for diagnosing sleep disorders.

Automatic identification of visual learning style in real time using raw electroencephalogram (EEG) is challenging. In this work, inspired by the powerful abilities of deep learning techniques, deep learning-based models are proposed to learn high-level feature representation for EEG visual learning identification. Existing computer-aided systems that use electroencephalograms and machine learning can reasonably assess learning styles. Despite their potential, offline processing is often necessary to eliminate artifacts and extract features, making these methods unsuitable for real-time applications. The dataset was chosen with 34 healthy subjects to measure their EEG signals during resting states (eyes open and eyes closed) and while performing learning tasks. The subjects displayed no prior knowledge of the animated educational content presented in video format. The paper presents an analysis of EEG signals measured during a resting state with closed eyes using three deep learning techniques: Long-term, short-term memory (LSTM), Long-term, short-term memory-convolutional neural network (LSTM-CNN), and Long-term, short-term memory-Fully convolutional neural network (LSTM-FCNN). The chosen techniques were based on their suitability for real-time applications with varying data lengths and the need for less computational time. The optimization of hypertuning parameters has enabled the identification of visual learners through the implementation of three techniques. LSTM-CNN technique has the highest average accuracy of 94%, a sensitivity of 80%, a specificity of 92%, and an F1 score of 94% when identifying the visual learning style of the student out of all three techniques. This research has shown that the most effective method is the deep learning-based LSTM-CNN technique, which accurately identifies a student's visual learning style.

Deep learning methods have advanced quickly in brain imaging analysis over the past few years, but they are usually restricted by the limited labeled data. Pre-trained model on unlabeled data has presented promising improvement in feature learning in many domains, such as natural language processing. However, this technique is under-explored in brain network analysis. In this paper, we focused on pre-training methods with Transformer networks to leverage existing unlabeled data for brain functional network classification. First, we proposed a Transformer-based neural network, named as BrainNPT, for brain functional network classification. The proposed method leveraged token as a classification embedding vector for the Transformer model to effectively capture the representation of brain networks. Second, we proposed a pre-training framework for BrainNPT model to leverage unlabeled brain network data to learn the structure information of brain functional networks. The results of classification experiments demonstrated the BrainNPT model without pre-training achieved the best performance with the state-of-the-art models, and the BrainNPT model with pre-training strongly outperformed the state-of-the-art models. The pre-training BrainNPT model improved 8.75% of accuracy compared with the model without pre-training. We further compared the pre-training strategies and the data augmentation methods, analyzed the influence of the parameters of the model, and explained the trained model.

Early diagnosis and intervention of depression promote complete recovery, with its traditional clinical assessments depending on the diagnostic scales, clinical experience of doctors and patient cooperation. Recent researches indicate that functional near-infrared spectroscopy (fNIRS) based on deep learning provides a promising approach to depression diagnosis. However, collecting large fNIRS datasets within a standard experimental paradigm remains challenging, limiting the applications of deep networks that require more data. To address these challenges, in this paper, we propose an fNIRS-driven depression recognition architecture based on cross-modal data augmentation (fCMDA), which converts fNIRS data into pseudo-sequence activation images. The approach incorporates a time-domain augmentation mechanism, including time warping and time masking, to generate diverse data. Additionally, we design a stimulation task-driven data pseudo-sequence method to map fNIRS data into pseudo-sequence activation images, facilitating the extraction of spatial-temporal, contextual and dynamic characteristics. Ultimately, we construct a depression recognition model based on deep classification networks using the imbalance loss function. Extensive experiments are performed on the two-class depression diagnosis and five-class depression severity recognition, which reveal impressive results with accuracy of 0.905 and 0.889, respectively. The fCMDA architecture provides a novel solution for effective depression recognition with limited data.

Major Depression Disorder (MDD) is a common yet destructive mental disorder that affects millions of people worldwide. Making early and accurate diagnosis of it is very meaningful. Recently, EEG, a non-invasive technique of recording spontaneous electrical activity of brains, has been widely used for MDD diagnosis. However, there are still some challenges in data quality and data size of EEG: (1) A large amount of noise is inevitable during EEG collection, making it difficult to extract discriminative features from raw EEG; (2) It is difficult to recruit a large number of subjects to collect sufficient and diverse data for model training. Both of the challenges cause the overfitting problem, especially for deep learning methods. In this paper, we propose DiffMDD, a diffusion-based deep learning framework for MDD diagnosis using EEG. Specifically, we extract more noise-irrelevant features to improve the model's robustness by designing the Forward Diffusion Noisy Training Module. Then we increase the size and diversity of data to help the model learn more generalized features by designing the Reverse Diffusion Data Augmentation Module. Finally, we re-train the classifier on the augmented dataset for MDD diagnosis. We conducted comprehensive experiments to test the overall performance and each module's effectiveness. The framework was validated on two public MDD diagnosis datasets, achieving the state-of-the-art performance.

The olfactory system enables humans to smell different odors, which are closely related to emotions. The high temporal resolution and non-invasiveness of Electroencephalogram (EEG) make it suitable to objectively study human preferences for odors. Effectively learning the temporal dynamics and spatial information from EEG is crucial for detecting odor-induced emotional valence. In this paper, we propose a deep learning architecture called Temporal Attention with Spatial Autoencoder Network (TASA) for predicting odor-induced emotions using EEG. TASA consists of a filter-bank layer, a spatial encoder, a time segmentation layer, a Long Short-Term Memory (LSTM) module, a multi-head self-attention (MSA) layer, and a fully connected layer. We improve upon the previous work by utilizing a two-phase learning framework, using the autoencoder module to learn the spatial information among electrodes by reconstructing the given input with a latent representation in the spatial dimension, which aims to minimize information loss compared to spatial filtering with CNN. The second improvement is inspired by the continuous nature of the olfactory process; we propose to use LSTM-MSA in TASA to capture its temporal dynamics by learning the intercorrelation among the time segments of the EEG. TASA is evaluated on an existing olfactory EEG dataset and compared with several existing deep learning architectures to demonstrate its effectiveness in predicting olfactory-triggered emotional responses. Interpretability analyses with DeepLIFT also suggest that TASA learns spatial-spectral features that are relevant to olfactory-induced emotion recognition.

Computational biomechanical analysis plays a pivotal role in understanding and improving human movements and physical functions. Although physics-based modeling methods can interpret the dynamic interaction between the neural drive to muscle dynamics and joint kinematics, they suffer from high computational latency. In recent years, data-driven methods have emerged as a promising alternative due to their fast execution speed, but label information is still required during training, which is not easy to acquire in practice. To tackle these issues, this paper presents a novel physics-informed deep learning method to predict muscle forces without any label information during model training. In addition, the proposed method could also identify personalized muscle-tendon parameters. To achieve this, the Hill muscle model-based forward dynamics is embedded into the deep neural network as the additional loss to further regulate the behavior of the deep neural network. Experimental validations on the wrist joint from six healthy subjects are performed, and a fully connected neural network (FNN) is selected to implement the proposed method. The predicted results of muscle forces show comparable or even lower root mean square error (RMSE) and higher coefficient of determination compared with baseline methods, which have to use the labeled surface electromyography (sEMG) signals, and it can also identify muscle-tendon parameters accurately, demonstrating the effectiveness of the proposed physics-informed deep learning method.

Brain-computer interfaces (BCIs) can enable direct communication with assistive devices by recording and decoding signals from the brain. To achieve high performance, many electrodes will be used, such as the recently developed invasive BCIs with channel numbers up to hundreds or even thousands. For those high-throughput BCIs, channel selection is important to reduce signal redundancy and invasiveness while maintaining decoding performance. However, such endeavour is rarely reported for invasive BCIs, especially those using deep learning methods. Two deep learning-based methods, referred to as Gumbel and STG, were proposed in this paper. They were evaluated using the Stereo-electroencephalography (SEEG) signals, and compared with three other methods, including manual selection, mutual information-based method (MI), and all channels (all channels without selection). The task is to classify the SEEG signals into five movements using channels selected by each method. When 10 channels were selected, the mean classification accuracies using Gumbel, STG (referred to as STG-10), manual selection, and MI selection were 65%, 60%, 60%, and 47%, respectively, whilst the accuracy was 59% using all channels (no selection). In addition, an investigation of the selected channels showed that Gumbel and STG have successfully identified the pre-central and post-central areas, which are closely related to motor control. Both Gumbel and STG successfully selected the informative channels in SEEG recordings while maintaining decoding accuracy. This study enables future high-throughput BCIs using deep learning methods, to identify useful channels and reduce computing and wireless transmission pressure.

The document is the table of contents for the May 2024 issue of the IEEE Transactions on Semiconductor Manufacturing. It includes papers on various topics such as yield modeling, analysis, and enhancement, advanced processing, advanced process control, equipment and automation technology, non-silicon materials, emerging areas, and environment, safety, and health. The articles cover subjects such as chip-scale chemical mechanical polishing, threshold voltage deviation monitoring, curvilinear standard cell design, fabrication of silicon nanocone arrays, improving the reliability of through silicon vias, energy consumption and carbon emission reduction in HVAC systems, low-k silicon dioxide synthesis, chemical mechanical polishing of diamond epitaxial layers, conditional variable selection based on deep learning, and eco-friendly dry-cleaning and diagnostics of silicon dioxide deposition chambers. The document also includes an announcement for the call for nominations for the 2024 IEEE EDS Early Career Award. [Extracted from the article]

Recent developments in the field of non-local attention (NLA) have led to a renewed interest in self-similarity-based single image super-resolution (SISR). Researchers usually use the NLA to explore non-local self-similarity (NSS) in SISR and achieve satisfactory reconstruction results. However, a surprising phenomenon that the reconstruction performance of the standard NLA is similar to that of the NLA with randomly selected regions prompted us to revisit NLA. In this paper, we first analyzed the attention map of the standard NLA from different perspectives and discovered that the resulting probability distribution always has full support for every local feature, which implies a statistical waste of assigning values to irrelevant non-local features, especially for SISR which needs to model long-range dependence with a large number of redundant non-local features. Based on these findings, we introduced a concise yet effective soft thresholding operation to obtain high-similarity-pass attention (HSPA), which is beneficial for generating a more compact and interpretable distribution. Furthermore, we derived some key properties of the soft thresholding operation that enable training our HSPA in an end-to-end manner. The HSPA can be integrated into existing deep SISR models as an efficient general building block. In addition, to demonstrate the effectiveness of the HSPA, we constructed a deep high-similarity-pass attention network (HSPAN) by integrating a few HSPAs in a simple backbone. Extensive experimental results demonstrate that HSPAN outperforms state-of-the-art approaches on both quantitative and qualitative evaluations. Our code and a pre-trained model were uploaded to GitHub (https://github.com/laoyangui/HSPAN) for validation. [ABSTRACT FROM AUTHOR]

Efficient point cloud compression is essential for applications like virtual and mixed reality, autonomous driving, and cultural heritage. This paper proposes a deep learning-based inter-frame encoding scheme for dynamic point cloud geometry compression. We propose a lossy geometry compression scheme that predicts the latent representation of the current frame using the previous frame by employing a novel feature space inter-prediction network. The proposed network utilizes sparse convolutions with hierarchical multiscale 3D feature learning to encode the current frame using the previous frame. The proposed method introduces a novel predictor network for motion compensation in the feature domain to map the latent representation of the previous frame to the coordinates of the current frame to predict the current frame’s feature embedding. The framework transmits the residual of the predicted features and the actual features by compressing them using a learned probabilistic factorized entropy model. At the receiver, the decoder hierarchically reconstructs the current frame by progressively rescaling the feature embedding. The proposed framework is compared to the state-of-the-art Video-based Point Cloud Compression (V-PCC) and Geometry-based Point Cloud Compression (G-PCC) schemes standardized by the Moving Picture Experts Group (MPEG). The proposed method achieves more than 88% BD-Rate (Bjøntegaard Delta Rate) reduction against G-PCCv20 Octree, more than 56% BD-Rate savings against G-PCCv20 Trisoup, more than 62% BD-Rate reduction against V-PCC intra-frame encoding mode, and more than 52% BD-Rate savings against V-PCC P-frame-based inter-frame encoding mode using HEVC. These significant performance gains are cross-checked and verified in the MPEG working group. [ABSTRACT FROM AUTHOR]

Thanks to the many submissions we have received, we can present this second part of our special issue on "AI-eXplained." In here, we continue our mission to demystify the intricate world of artificial intelligence and make it accessible to a broader audience. As AI continues to evolve and integrate into various aspects of our lives, it becomes increasingly important to bridge the gap between experts and those eager to understand the inner workings of AI systems. Our goal remains unchanged—to present AI concepts in a comprehensible, engaging, and interactive manner, empowering our readers to explore and grasp the enchanting world of AI. [ABSTRACT FROM AUTHOR]

Biometric identification technologies are crucial for enhancing security through reliable personal authentication methods. Among these modalities, finger knuckle biometrics stands out for its distinctive and consistent features, offering a valuable alternative to more commonly used biometric traits. Unlike fingerprints, which are easily captured from the surface of the skin, knuckle prints present a unique challenge. Knuckle prints are not as readily accessible from surface scans due to their position and the intricacy of their features, which require specialized techniques for accurate capture and recognition. The paper comprehensively reviews the evolution from traditional geometric methods to advanced deep learning techniques in finger knuckle recognition. Our review covers both unimodal and multimodal approaches, discussing various recognition strategies and their effectiveness. We also discussed the performance of knuckle biometric systems using metrics such as accuracy, False Acceptance Rate (FAR), False Rejection Rate (FRR), and Equal Error Rate (EER). The paper also highlights the importance of publicly available knuckle datasets, which are essential for developing and evaluating FKP biometric systems. These datasets enable researchers to benchmark and improve recognition algorithms. This review is aimed at researchers, practitioners, and academics interested in biometric technologies, offering insights into current advancements and future directions in finger knuckle biometrics.

The synergy between software engineering (SE) and artificial intelligence (AI) catalyzes software development, as numerous recent studies illustrate an intensified intersection between these domains. This systematic literature review examines the integration of AI techniques or methodologies across SE phases and related activities spanning from 2013 to 2023, resulting in the selection of 110 research papers. Investigating the profound influence of AI techniques, including machine learning, deep learning, natural language processing, optimization algorithms, and expert systems, across various SE phases—such as planning, requirement engineering, design, development, testing, deployment, and maintenance—is the focal point of this study. Notably, the extensive adoption of machine learning and deep learning algorithms in the development and testing phases has enhanced software quality through defect prediction, code recommendation, and vulnerability detection initiatives. Furthermore, natural language processing’s role in automating requirements classification and sentiment analysis has streamlined SE practices. Optimization algorithms have also demonstrated efficacy in refining SE activities such as feature location and software repair action predictions, augmenting precision and efficiency in maintenance endeavors. Prospective research emphasizes the imperative of interpretable AI models and the exploration of novel AI paradigms, including explainable AI and reinforcement learning, to promote ethical and efficient software development practices. This paper fills the gap identified in AI techniques dedicated to improving SE phases. The review concludes that AI in SE is revolutionizing the discipline, enhancing software quality, efficiency, and innovation, with ongoing efforts targeting the mitigation of identified limitations and the augmentation of AI capabilities for intelligent and dependable SE.

Shared gradients are extensively utilized for safeguarding the privacy of training data. However, an increasing body of research is uncovering that the gradients or model parameters transmitted in distributed systems may also leak users’ private information. A majority of these studies are predicated on intercepting the data transmitted by individual users to derive their private information. A smaller portion of research can extrapolate a multitude of users’ private data through the average gradients passed by the server. These investigations have analyzed information leakage in image and text domains, yet have not explored the leakage issues inherent within recommendation systems operating in distributed environments. Furthermore, the impact of batch size and model parameters on the degree of leakage has not been sufficiently analyzed. This paper proposes that within recommendation systems, it is feasible to infer the encapsulated user privacy data by receiving average gradients or model parameters provided by the server. Additionally, the paper evaluates the extent to which various parameters within the system impact the leakage and presents corresponding defensive measures while assessing their efficacy.

Plant pest and disease management, especially in the early stages of infestation, is a critical challenge that poses significant threats and has potential to devastate agricultural crops, causing total yield loss and food insecurity. Traditional inspection methods are time-consuming and prone to errors due to limited labor expertise. Therefore, to tackle these challenges, harnessing advanced technologies such as artificial intelligence (AI), Machine Learning/Deep Learning (ML/DL), and Internet of Things (IoT) is essential for managing and mitigating agriculture hazards. This research presents a comprehensive review of the state-of-the-art DL architectures integrated with IoT-based systems applied to plant pest and disease detection (PPDD) by investigating different potential approaches that have been employed using DL and IoT up to the year 2024 to address challenges in agriculture. Convolutional Neural Network (CNN) architectures for image recognition, object detection, and their integration with IoT, embedded into mobile devices and unmanned aerial vehicles (UAV) are explored. Moreover, the research discusses the advantages and limitations of these techniques, emphasizing their architecture design, efficiency and accuracy. The findings demonstrate that there is a tradeoff between robustness and complexity among existing techniques, and authors recommend future trends aimed at creating robust models with fewer parameters that are more accurate and easily implementable on small IoT-based and portable devices suitable for in-field and real-time applications. Furthermore, while existing review papers discuss either DL or IoT separately, this research paper uniquely focuses on their combined models, providing a comprehensive overview of the synergistic potential of leveraging IoT-driven technologies alongside advanced DL algorithms to ease the task of researchers in the field of precision agriculture particularly in PPDD.

To enhance the effective use of teaching video information in digital campuses, provide timely and accurate assessment of students’ classroom learning status, and offer technical support for teaching management, this paper introduces an improved SSD (Single Shot MultiBox Detector) object detection algorithm for intelligent analysis of students’ real classroom behavior. This article examines classroom teaching videos from four months of a semester at a certain university, collecting over 10,446 1080p video frames to create a dataset for annotation and training. To address the poor detection performance of the SSD model when identifying four behavior states—listening, sleeping, playing, and writing—of students in combined classes due to object occlusion and complex classroom environments, this paper introduces ResNet residual modules and a dual attention mechanism into the VGG-16 (Visual Geometry Group-16) network. This enhancement improves the model’s ability to detect student states in video frames under conditions of object occlusion. An FPN (Feature Pyramid Network) fusion model is constructed to address the low recognition accuracy of the SSD model for small objects in the back rows, thereby improving image recognition accuracy. The experimental results show that the improved SSD model has improved feature extraction ability and small object recognition accuracy compared to the original SSD model, with mAP increasing from 51.62% to 63.75%. The research results of this project can provide real-time information reference for university teaching management, help teachers understand students’ learning status promptly, and adjust teaching methods on time to improve teaching quality and classroom effectiveness.

Millimeter-wave (mmWave) frequencies ranging from 30 to 300 GHz offer vast bandwidth and high data transmission rates, making them ideal for high-throughput applications and the expanding Internet of Things (IoT). However, mmWave implementation faces challenges such as narrow beams, susceptibility to blockage, and rapid channel fluctuations due to user mobility. To address these issues, non-orthogonal multiple access (NOMA) is employed, enhancing spectral efficiency by allowing multiple users to share the same frequency resources at different power levels. This paper focuses on Power-Domain NOMA (PD-NOMA), a variant of NOMA that allocates different power levels to users sharing the same frequency resources. Although other forms of NOMA, such as Code-Domain NOMA (CD-NOMA) and Cooperative NOMA, exist, our discussion will primarily focus on PD-NOMA due to its practical application in mmWave-NOMA networks. The integration of mmWave and NOMA presents both significant opportunities and complex challenges, particularly in resource management. This combination aims to leverage mmWave’s high bandwidth and NOMA’s efficient resources utilization to overcome physical layer limitations and enhance network performance. Traditional methods struggle with optimizing those resources like power levels, bandwidth, beam directions, and user pairing. Deep learning (DL) presents a promising solution by learning optimal resource allocation from data. This paper reviews current and future DL applications in five key areas: power allocation, energy efficiency, user association, bandwidth allocation, and subcarrier allocation in mmWave-NOMA networks. It also highlights available open-source datasets, code, and DL frameworks supporting these developments and discusses key research directions. The structure of the paper is organized as follows: first, background knowledge on mmWave and NOMA systems is presented, followed by an in-depth review of DL applications for resource management. Finally, we conclude with challenges and future research directions in this evolving field.

The accurate estimation of Li-ion battery (LIB) states such as State of Charge (SOC), State of Health (SOH), and State of Power (SOP) plays a pivotal role in the efficient operation of Electric Vehicles (EVs). These parameters can impact the battery’s health, driving range, and overall vehicle performance. Transformer-based artificial neural networks have shown impressive results in natural language processing (NLP) and estimation problems of many other domains. This paper presents an intensive study on the capabilities of various Transformer-based models in estimating the SOC and SOH of LIBs, the SOP is obtained based on the estimated SOC. This paper provides the following key original contributions: 1) the application of the Informer and Reformer variants of the Transformer model for the first time for SOH estimation of LIBs in EVs, 2) studying the effect of inverted embedding of iTransformers, a modified architecture of the transformers, on SOC and SOH estimation, inversion is performed on the Informer and Reformer as well; 3) applying a simple feature extraction method using partial discharge cycles for SOH estimation with Transformer-based models; 4) a new robust method is proposed for SOC estimation based on a 2-Encoder-Transformer with a one-dimensional convolutional neural network (1D-CNN) architecture; 5) the various architectures are trained, validated and tested on two real-world datasets comprising various driving scenarios and battery conditions. Comparative analysis with various deep learning architectures show impressive accuracy for estimating the SOC and SOH, leading to better SOP calculation.

Gastrointestinal (GI) diseases are most common worldwide and the death rate can be reduced by early detection. Endoscopy is widely regarded as the gold standard for diagnosing and managing digestive disorders, affecting both the upper and lower GI tracts. Endoscopy is performed to uncover biopsy tissues used to check the presence of cancerous or benign cells, Helicobacter pylori (H. pylori) infection, or perform colonoscopy in case of the removal of polyps. A systematic review was conducted on databases like PubMed, Scopus, Google Scholar, and IEEE Explore, including research papers published up to May 2023, through the systematic search, 33 papers were identified. This review offers valuable insights to physicians and technological guidance to future researchers by examining GI tract diseases. It provides a detailed analysis of Machine learning (ML) techniques like preprocessing, segmentation, feature extraction, and classification. Additionally, Deep Learning (DL) approaches like transfer learning (TL), Convolution Neural Networks (CNN), optimization, transformer, and reinforcement learning have been analyzed for GI diagnosis. The DL approach has increased its use in GI diseases and CNN was the most commonly used architecture. Lastly, the review highlights the research published in the specialized GI fields and provides technological suggestions and insights for future research prospects. Overall, this study broadens the body of knowledge regarding the existing Artificial Intelligence (AI) techniques in gastroenterology as a manual for creating and assessing AI models.

Advancements in deep learning enable sophisticated algorithms to learn from vast data sets, driving innovation and efficiency in power systems. The global push for electrification has increased focus on deep learning, leading to numerous recent publications in this field. This paper presents a bibliometric analysis and future trends of deep learning in power systems, aiming to identify its fundamental characteristics and summarize the research hot topics and future trends. This study used the Scopus database with the keywords “deep learning” and “power systems”, yielding 20,202 publications from 1970 to 2023. Over 37,600 authors contributed, averaging 12 citations per paper. Keyword trends show traditional deep learning techniques like LSTMs and CNNs are widely used in power systems, while newer methods like advanced reinforcement learning, graph neural networks, and physics-informed neural networks are emerging, promising advancements in optimal power flow, V2G integrations, and grid resilience. Further analysis highlights ubiquitous deep learning applications like load forecasting and power quality analysis in power systems. Emerging topics include microgrid optimization and electric vehicle charging demand prediction, with growing interest in IoT management, digital twins, and cybersecurity. Future research will focus on self-healing grids, optimal power flow, and energy trading for improved reliability and security.

Insider threats cause greater losses than external attacks, prompting organizations to invest in detection systems. However, there exist challenges: 1) Security and privacy concerns prevent data sharing, making it difficult to train robust models and identify new attacks. 2) The diversity and uniqueness of organizations require localized models, as a universal solution could be more effective. 3) High resource costs, delays, and data security concerns complicate building effective detection systems. This paper introduces FedITD, a flexible, hierarchy, and federated framework with local real-time detection systems, combining Large Language Models (LLM), Federated Learning (FL), Parameter Efficient Tuning (PETuning), and Transfer Learning (TF) for insider threat detection. FedITD uses FL to protect privacy while indirect integrating client information and employs PETuning methods (Adapter, BitFit, LoRA) with LLMs (BERT, RoBERTa, XLNet, DistilBERT) to reduce resource use and time delay. FedITD customizes client models and optimizes performance via transfer learning without central data transfer, further enhancing the detection of new attacks. FedITD outperforms other federated learning methods and its performance is very close to the best centrally trained method. Extensive experiment results show FedITD’s superior performance, adaptability to varied data, and reduction of resource costs, achieving an optimal balance in detection capabilities across source data, unlabeled local data, and global data. Alternative PETuning implementations are also explored in this paper.

While recognition of sign language alphabets and isolated words has matured in recent years, recognition of sign language sentences, or continuous signing, is still a research topic of interest in computer vision, especially in signer independent mode of recognition. Existing state-of-the-art solutions in the continuous Arabic Sign Language Recognition (ArSLR) are promising; however, when implemented in signer independent mode, the accuracies drop noticeably. In this paper, we propose a solution for recognizing continuous Arabic signing in signer dependent and independent modes through the use of motion images with positioning information. Initially, sign videos are converted into several motion images, each emphasizing a different segment of the sentence. This is achieved by a weighted sum of residual images after applying optical flow and motion compensation. Each motion image is composed of the whole sentence video, hence putting the emphasized segment into context in terms of previous and successive sign words. Thereafter, hand-crafted features are calculated from each resultant image, including numerical summaries of the horizontal, vertical and diagonal profiles. With such features, the architecture used for model generation and testing is simplified, where it consists of a single Bi-LSTM layer followed by dropout, softmax, and classification layers. This paper makes use of a recent Arabic Sign Language dataset known as ArabSign. The dataset is composed of 6 signers and 93 words arranged into 50 sentences with 30 repetitions each. Experimental results revealed that the proposed solution is suitable for signer dependent and signer independent modes of continuous sign language recognition. Using a Leave-One-Signer-Out policy, the proposed solution achieved word-recognition rates of 99.8% and 75.3%, respectively. These results noticeably surpass relevant state-of-the-art solutions.

With advancements in technology, deep learning has become increasingly widespread, particularly in fields like robot control, computer vision, and autonomous driving. In these areas, obtaining pose information of target objects, especially their spatial location, is crucial for robot grasping tasks. Although many effective implementations of six degrees of freedom (6D) pose estimation methods based on RGB images exist, challenges in this domain persist. This paper provides a comprehensive review of traditional 6D pose estimation methods, deep learning approaches, and point cloud techniques by analyzing their advantages and disadvantages. It also discusses evaluation metrics and performance on common datasets for 6D pose estimation. Furthermore, the paper offers a theoretical foundation for robot grasping and explores future directions for 6D pose estimation. Finally, it summarizes the current state and development trends of 6D pose estimation, aiming to help researchers better understand and learn about this field.

Analyzing and understanding emotional expressions in user comments is a crucial and complex task in business. Conducting text sentiment analysis is of great significance. This paper constructs a novel DIBTBL model that integrates an extended sentiment dictionary, an improved swarm intelligence algorithm, and deep learning techniques to accurately capture and analyze user emotions. Firstly, this paper expands the sentiment dictionary to extract emotion feature words from the review text. Secondly, the BERT model embeds these emotional feature words and pre-processed text into high-dimensional semantic space to obtain richer semantic representations and improve sentiment classification performance. Then, the TextCNN-BiLSTM feature extraction model is established to balance the grasping ability of local and global features. Fourthly, this paper innovatively improves the swarm intelligence algorithm BWO to optimize the parameters of the TextCNN-BiLSTM. Finally, MLP is employed for sentiment classification. The experimental data is crawled from Ctrip, China’s largest hotel booking website. In the comparative experiment, the proposed model achieves a higher accuracy than TBL, DTBL, BTBL, and IBTBL by 2.94%, 2.44%, 1.64%, and 0.66%, respectively. In addition, we compare the proposed model with seven advanced models in the open dataset waimai_10k. The experimental results indicate that this model outperforms all the other models, with an average improvement in accuracy of 8.21%. The study offers precise insights into user sentiment, assisting companies in better understanding and meeting customer needs.

With the rapid advancement of technology, deep learning has demonstrated significant application potential in the fields of video surveillance and behavior recognition. This paper explores the application of an improved Convolutional Neural Network (CNN)-based deep learning model in video behavior recognition and discusses privacy protection strategies. By incorporating multi-scale feature fusion mechanisms, spatiotemporal attention mechanisms, and Long Short-Term Memory (LSTM) networks, the designed deep learning model achieved notable performance improvements on the UCF-101 dataset, with an accuracy rate of 95.8%, precision of 96.5%, recall of 95.2%, and an F1 score of 95.8%. Additionally, this paper outlines a comprehensive privacy protection strategy that includes data anonymization, encrypted transmission, and access control, effectively safeguarding personal privacy. Experimental results indicate that although the privacy protection measures led to an increase in data processing time by approximately 40% and a decrease in model performance by about 2.5%, they significantly enhanced data security. This study provides valuable insights and references for the application of deep learning in video surveillance and behavior recognition, as well as its associated privacy protection.

Artificial intelligence (AI) systems, based on neural networks are becoming ubiquitous. However, in many cases, their application is constrained by the lack of interpretability. One of the ways of overcoming this limitation is neuro-symbolic approaches, where the neural network is complemented by some symbolic structures, expressing existing knowledge and making it understandable to humans. Such approaches can make predictions more understandable and interpretable (due to the connection to human-understandable symbolic structures). Still, they can also have an additional advantage by allowing models to be trained with less data (due to leveraging prior knowledge). This paper focuses on a subset of neuro-symbolic approaches, where domain ontologies play the role of symbolic structures. The paper discusses the existing methods to build ontology-aware explainable neural networks and ways to leverage ontologies in forming the explanation. Based on the analysis, it proposes a computational framework for building ontology-aware self-explaining neural networks. The proposed framework allows several specializations, and it is shown that these specializations allow one to improve the prediction quality. Finally, the paper presents the results of a user study, showcasing that ontology-based explanations can improve the understandability of an AI model and the efficiency of human-AI interaction.

Road Condition Monitoring is a critical task for the management and maintenance of the pavement network infrastructure by the authorities. In recent years, the application of Artificial Intelligence (AI) techniques in this domain has experienced a significant growth, driven by the continuous advancements in AI algorithms. This paper presents a comprehensive review of the latest developments in Road Condition Monitoring approaches using AI methods, with a particular focus on Deep Learning techniques, covering works published from 2020 onwards. It highlights novel approaches that have not been thoroughly explored in previous literature reviews. The literature review categorizes studies based on the type of signal data, distinguishing between acoustic, vibrational, and vision-based approaches. For each data type, the paper examines and discuss the most recent advancements and improvements achieved through AI techniques. Additionally, it provides an overview of future directions and identifying key challenges that remain open in the field. In conclusion, relatively few studies have focused on the analysis of acoustic data, although some studies have reported promising results. Methods based on vibrational data typically integrate feature extraction in frequency and wavelet domain with Convolutional Neural Networks or Long Short-Term Memory Networks. Meanwhile, vision-based methods have experienced significant improvements, driven by the constant evolution of Deep Learning architectures. A total of 173 research articles are summarized across 10 tables.

Early diagnosis of citrus diseases directly affects the yield and quality of citrus cultivation, and a citrus disease classification model based on improved ConvNeXt is proposed to address the problems of high cost and low efficiency of traditional citrus disease detection methods. Firstly, the attention mechanism is changed to a parallel connection in the convolutional block attention module, and the attention mechanism is incorporated into ConvNeXt, which improves the model’s ability of feature extraction, makes the model focus more on lesion features, and suppresses the interference of background information. Secondly, the multi-scale feature fusion module is incorporated to improve the model’s adaptability to disease features at different scales and improve the network classification performance. Finally, the transfer learning method is used to conduct pre-training with ImageNet weight information to reduce the impact of insufficient samples. The experimental results show that on the self-constructed complex background citrus disease dataset, the model in this paper achieves an average accuracy of 98.07%, which is improved by 2.9% compared with the original model, and the comprehensive performance is significantly better than that of VGG, AlexNet, and other models. The model in this paper improves the performance of disease classification and provides a theoretical basis for intelligent classification of citrus diseases.

Emotion recognition from speech signals plays a crucial role in Human-Machine Interaction (HMI), particularly in the development of applications such as affective computing and interactive systems. This review seeks to provide an in-depth examination of current methodologies in speech emotion recognition (SER), with a focus on databases, feature extraction techniques, and classification models. It has been done in the past using low-level descriptors (LLDs) like Mel-Frequency Cepstral Coefficients (MFCCs), linear predictive coding (LPC), and pitch-based features in methods like Support Vector Machines (SVM), Random Forests (RF), and Gaussian Mixture Models (GMM). But the development of deep learning techniques has completely changed the field. Models like convolutional neural networks (CNNs) and long short-term memory (LSTM) networks have shown that they are better at capturing the complex temporal and spectral features of speech. This paper reviews prominent speech emotion datasets, exploring their linguistic diversity, annotation processes, and emotional labels. It also analyzes the efficacy of different speech features and classifiers in handling challenges such as data imbalance, limited data availability, and cross-lingual variations. The review highlights the need for future work to address real-time processing, context-sensitive emotion detection, and the integration of multi-modal data to enhance the performance of SER systems. By consolidating recent advancements and identifying areas for further research, this paper aims to provide a clearer path for optimizing feature extraction and classification techniques in the field of emotion recognition.

Reliance on social networks has become an integral part of modern daily activities. Social networks are crowded with vast numbers of comments, opinions, and beliefs about different aspects of people’s daily lives. Opinions can directly express positive or negative perspectives toward a specific entity or its aspects, or they can be comparative opinions that present the differences or similarities between two or more entities, such as “product A is better than product B.” Comparative opinions have significant potential to offer new opportunities across various fields such as marketing, education, and e-commerce. Thus, analyzing comparative opinions is essential. Indeed, comparative opinion mining has been applied in several languages, including Arabic. Nevertheless, comparative opinion mining in Arabic is still in its early stages and requires more contributions and investigations. Therefore, this work introduces the ACOM approach for Arabic Comparative Opinion Mining. First, we collected the ACOM corpus in Arabic from the X platform, focusing on comparative opinions in the technology domain. This paper provides a benchmark comparison of several deep-learning models by employing different approaches, including Long Short-Term Memory (LSTM), Bidirectional LSTMs (BiLSTM), and Convolutional Neural Networks (CNN). In the deep learning model, the generated representation vector was used to construct the embedding layer as the first layer. To this end, we adopted two pretrained word embedding techniques to investigate their effectiveness on the ACOM corpus: Word2vec and Bidirectional Encoder Representations from Transformers (BERT). The main highlight of this paper is leveraging the most advanced innovation in AI, the GPT-3 model, as a classifier model. Our experimental results show that BERT and BiLSTM achieved impressive performance on the ACOM task with 91% accuracy and a 90% F1-score. Additionally, this paper discusses the impact of fine-tuning specific hyperparameters by providing a comprehensive examination.

The unprecedented progress in computational technologies led to a substantial proliferation of artificial intelligence applications, notably in the era of big data and IoT devices. In the face of exponential data growth and complex computations, conventional computing encounters substantial obstacles pertaining to energy efficiency, computational speed, and area. Due to the diminishing advantages of technology scaling and increased demands from computing workloads, novel design techniques are required to increase performance and decrease power consumption. Approximate computing, nowadays considered a promising paradigm, achieves considerable improvements in overhead cost reduction (i.e., energy, area, and latency) at the expense of a modest (i.e., still acceptable) deterioration in application accuracy. Therefore, approximate computing at different levels (Data, Circuit, Architecture, and Software) has been attracted by the research and industrial communities. This paper presents a comprehensive review of the major research areas of different levels of approximate computing by exploring their underlying principles, potential benefits, and associated trade-offs. This is a burgeoning field that seeks to balance computational efficiency with acceptable accuracy. The paper highlights opportunities where these techniques can be effectively applied, such as in applications where perfect accuracy is not a strict requirement. This paper presents assessments of applying approximate computing techniques in various applications, especially machine learning algorithms (ML) and IoT. Furthermore, this review underscores the challenges encountered in implementing approximate computing techniques and highlights potential future research avenues. The anticipation is that this survey will stimulate further discourse and underscore the necessity for continued research and development to fully exploit the potential of approximate computing.

Diabetic Retinopathy (DR) is the most common complication of Diabetes Mellitus and can lead to blindness if not detected early. Since DR is often asymptomatic in its early stage, timely diagnosis is crucial. Artificial Intelligence (AI) has the potential to facilitate early disease detection and treatment, but its implementation in the medical field raises significant privacy concerns. The sensitive nature of healthcare data, which includes personal information and medical history, makes data privacy a critical issue. This paper explores the implementation of AI models to predict DR risks while incorporating common defense algorithms to enhance data privacy. An unstructured dataset, specifically the DDR dataset, was used to train Deep Learning (DL) models. Two families of DL models, ResNets and DenseNets, were trained and evaluated based on the performance metrics. ResNet 50 and DenseNet 169 demonstrated superior performance and were selected for further privacy enhancement using encryption. The results indicated that privacy-preserving methods, particularly encryption, did not significantly impact the model performance. In summary, this paper highlights the potential of privacy-preserving AI in predicting the risks of DR.

Recent advancements in robotics have pushed the development of active exoskeletons and orthoses for assistive, augmentative, and rehabilitative purposes. Deep Learning approaches, particularly in motion analysis and prediction, hold promise for enhanced control of lower limb exoskeletons, offering the potential for improved outcomes in assistive and rehabilitative interventions. This review paper explores recent advancements in deep learning approaches for controlling lower limb exoskeletons. The study encompasses papers published from 2018 to the present, focusing on various aspects of deep learning models, including recognition, prediction, and other related tasks. The paper includes 103 papers covering various tasks and parameters like the gait phase, kinematics, and kinetics of the lower limb exoskeleton. Each aspect is thoroughly examined, highlighting the parameters utilized in the respective models. Moreover, the results obtained from these approaches are evaluated and compared against classical control strategies, shedding light on their effectiveness and potential benefits. The review also addresses the limitations of current deep learning techniques in lower limb exoskeleton control and outlines potential avenues for future research and improvement. By consolidating the latest findings and advancements in this field, this review paper provides valuable insights into the application of deep learning in the control of lower limb exoskeletons, paving the way for enhanced rehabilitation and assistive technologies in the future.

The advent of the sixth generation of mobile communications (6G) ushers in an era of heightened demand for advanced network intelligence to tackle the challenges of an expanding network landscape and increasing service demands. Deep Learning (DL), as a crucial technique for instilling intelligence into 6G, has demonstrated powerful and promising development. This paper provides a comprehensive overview of the pivotal role of DL in 6G, exploring the myriad opportunities and challenges that arise. Firstly, we present a detailed vision for DL in 6G, emphasizing areas such as adaptive resource allocation, intelligent network management, robust signal processing, ubiquitous edge intelligence, and endogenous security. Secondly, this paper reviews how DL models leverage their unique learning capabilities to solve complex service demands in 6G. The models discussed include Convolutional Neural Networks (CNN), Generative Adversarial Networks (GAN), Graph Neural Networks (GNN), Deep Reinforcement Learning (DRL), Transformer, Federated Learning (FL), and Meta Learning. Additionally, we examine the specific challenges each DL model faces within the 6G context. Moreover, we delve into the rapidly evolving field of Artificial Intelligence Generated Content (AIGC), examining its development and impact within the 6G framework. Finally, this paper culminates in a detailed discussion of ten critical open problems in integrating DL with 6G, setting the stage for future research and development in this field.

Over the past few years, there has been notable advancement in the field of Quantified Gait Analysis (QGA), thanks to machine learning techniques. QGA and gait prediction are areas where Deep learning (DL) techniques are gaining popularity. There has been a significant amount of attention from the scientific community on the application of gait analysis in various fields. Based on our understanding, there is a noticeable absence of a comprehensive review and current understanding of gait analysis utilizing DL and Multi-task learning (MTL) models. Therefore, this paper provides a comprehensive assessment of the current application of DL algorithms for QGA. The study takes a systematic approach to explore this topic in depth. We conducted a thorough search of three databases, namely Web of Science, IEEEXplore, and Scopus, to identify relevant papers published from 1989 to October 2023. A total of 55 papers were considered eligible and included in this review. Approximately 46% of the studies that were identified utilized classification models to categorize gait phases and locomotion modes. Additionally, a significant portion of the studies (45%) utilized regression models to estimate and predict various kinematic and kinetic parameters, including joint angles, trajectories, moments, and torques. Interestingly, a notable 9% of the studies employed the use of MTL techniques in the realm of DL for gait analysis. We have also provided information on the most commonly utilized datasets for QGA.

This paper aims to find that predictive performance is better when earning management predictions using deep learning technology, including litigation case information filed with companies. Unlike previous accounting forensics studies, this study designed a basic earning management prediction model using related financial variables by referring to various previous studies on earning management in accounting. To present more objective and reliable verification results, this study used two measures of accrual earning management (AEM) as proxy variables for corporate earning management. It used four deep-learning classifiers: RNN, GRU, LSTM, and Transformers. In addition, before predicting earning management through deep learning analysis, regression analysis proved that the number and amount of litigation cases filed against the company significantly reduced accrual earning management. In this study, the number of litigation cases filed with the company and the amount of litigation were used as information on the company’s litigation cases. The main findings of this paper are as follows. First, as expected, when predicting the earning management level, including the company’s litigation case information, accuracy, recall, precision, AUC (area under the ROC curve), and F1 score, all showed high overall predictive performance. Second, overall predictive accuracy was higher when predicting the level of earning management, including the number of litigation cases, than when predicting the level of earning management, including the number of litigation cases. Overall predictive performance was the highest when predicting the earning management level, including the number and amount of litigation cases. Third, among the four classifiers of deep learning, Transformers’ predictive performance was the best, followed by LSTM, GRU, and RNN.

Epileptic seizures can be dangerous as they cause sudden and uncontrolled electrical activity in the brain which can lead to injuries if one falls or loss of control over physical functions. To mitigate these risks, machine learning and deep learning algorithms are being developed to anticipate seizure occurrences. Accurate prediction of seizures could enable patients to adopt preventive strategies or initiate medical interventions to halt seizures, thereby minimizing injuries and enhancing safety for individuals afflicted with epilepsy. This paper aims to combine neural networks and Ensemble learning to enhance the accuracy of diagnosing epileptic seizures. By leveraging the strengths of both techniques, the precision in seizure diagnosis can be significantly improved. It also improves the evaluation metrics used in machine learning methodologies for a more comprehensive assessment of diagnostic outcomes. This approach ensures a thorough understanding of the effectiveness of the proposed approach. In this paper, a model with a supreme precision rate is developed to detect epileptic seizures with the help of EEG signals. This study uses an ensemble method, which employs several algorithms, for instance XGB, SVM, RF, and BiLSTM. The used dataset is open access from Bonn University. The proposed methodology reached 98.52% accuracy, 97.37% precision, 95.29% recall, and 96.32% F1-score, respectively.

The use of helmets is crucial for safeguarding the lives of construction workers. In the construction sector, computer vision technology is extensively employed to detect and monitor the correct usage of helmets by workers. Currently, there are three classical types of helmet detection algorithms: digital image processing, convolutional neural network (CNN), and Transformer. Digital images are based on manual processing of the features, which proves to be inefficient and lacks robustness. CNN exhibits high accuracy but lacks robustness, which limits its effectiveness in complex environments. This paper proposes an algorithm called the Highly Robust Helmet Detection Algorithm (HRHD), designed to attain precise detection of helmet usage at construction sites with varying conditions. The proposed model leverages the YOLO v8s architecture and incorporates the Coordinate Attention module to enhance the model’s focus on important features. It also introduces the Transformer structure to extract global features, and employs the RepConv module to diminish the model’s computational demands, thus achieving a balance between inference speed and detection accuracy. The experiments demonstrate that the proposed model in this paper significantly improves the accuracy compared to YOLO v10 and YOLO v8s. Additionally, the model maintains a rapid inference rate, suggesting substantial potential for application within the construction engineering domain.

Aiming at the incomplete data and difficult prediction in the prediction of firewater system water level, a data filling method is proposed based on the reinforcement learning approach and a deep learning (DL) prediction method is studied based on the long-term short-term memory (LSTM) model in this paper. Firstly, a reinforcement learning-based K-nearest neighbor (KNN) algorithm is designed for the data incompleteness that occurs in the firewater level. The parameters of the KNN algorithm are optimized through the reinforcement learning to enhance the accuracy of data filling. Secondly, to compensate for the shortcomings of the traditional firewater pool level prediction methods, a DL prediction method based on the LSTM network approach is proposed by using the complete data. Through the LSTM three gate functions, the previous information is rounded to avoid causing gradient explosion, rounding the method is more stable and accurate. Finally, the simulation results show that the proposed method in this paper can effectively predict incomplete water level information.

Sign language (SL) is a vital mode of communication, bridging the gap between the hearing impaired and hearing communities. However, SL, despite its paramount importance, has received relatively limited attention from researchers. Its unique structural characteristics, distinct from those of natural languages, present novel challenges that require innovative solutions. Remarkable technological advancements, notably in Artificial Intelligence (AI) and machine learning, offer promising avenues for automated Sign Language translation Systems (SLTS). This review study addresses the crucial need for a comprehensive synthesis of existing research by systematically examining and evaluating the progress made in SLTS. By analyzing 58 research papers, with a particular emphasis on the most frequently cited papers from each year up to 2023, we shed light on the field’s current state, identifying key advancements and challenges. This review followed a systematic approach based on clear guidelines. The methodology involved defining research questions, formulating queries, selecting studies based on clear criteria, and extracting pertinent information to address the research objectives. This review found that deep learning techniques, such as convolutional and recurrent neural networks, have shown high accuracy in sign language recognition, and their performance in recognizing the variety of signs has steadily improved over time. Additionally, integrating non-manual features has proven pivotal in enhancing recognition accuracy. Future research should refine advanced deep learning models and integrate non-manual features to improve system accuracy and applicability. These ongoing advancements hold the potential to revolutionize communication and break down barriers for individuals who rely on sign language as their primary mode of communication.

Hyperspectral imagery, capturing information beyond the visible spectrum, is essential for many remote sensing applications. This paper introduces a novel approach utilizing a 3D Convolutional Neural Network (CNN) enhanced with self-attention mechanism to address the limitations of traditional architectures in hyperspectral image classification. The main problem tackled is the effective extraction of both spatial and spectral features from complex hyperspectral datasets. Our proposed model simultaneously analyzes spatial and spectral dimensions with improved accuracy and adaptability through self-attention. We systematically evaluate the performance of 1D CNN, 2D CNN, and 3D CNN models, alongside Multi-Column Neural Network (MCNN) and Hybrid Spectral Network (Hybrid SN) models, comparing them to our proposed approach. The evaluation, conducted on the Indian Pines and Salinas datasets, highlights significant improvements in classification accuracy and adaptability. Specifically, the proposed 3D CNN with self-attention achieves accuracy values of 98.45% and 91.32% on the Indian Pines and Salinas datasets, respectively. This paper provides critical insights for data scientists and remote sensing experts, contributing to informed decision-making in selecting CNN models tailored to specific application requirements and advancing the field of hyperspectral image classification (HSIC). Furthermore, our approach shows improved handling of the complex spectral variability and spatial heterogeneity inherent in hyperspectral data.

Effective cardiovascular health monitoring relies on precise electrocardiogram (ECG) analysis for early diagnosis and treatment of heart conditions. Recent advancements in deep learning, particularly through Convolutional Neural Networks (CNNs), have significantly enhanced the automation, accuracy, and personalization of ECG analysis. This review targets both medical professionals and a broader audience interested in deep learning applications. Our work explores the evolution of deep learning techniques in ECG analysis, from early CNN applications to current innovations in real-time processing and privacy-preserving methods. The paper discusses various deep learning models, including hybrid models, Recurrent Neural Networks (RNNs), and attention mechanisms, and their impact on diagnostic accuracy for diseases like myocardial infarction. Additionally, our paper examines ECG-based authentication systems, addressing challenges related to security and privacy, and highlighting recent technological advancements. By providing a detailed overview of these developments, the review offers valuable insights into future directions for deep learning in cardiovascular health monitoring and ECG-based authentication.