Search

Applying deep learning, especially time series neural network, to stock market prediction, has become one of the important applications in the quantitative finance field. However, due to the multi-correlation and volatility of the stock market, how to timely and accurately predict it has become a challenging issue. In order to cope with this challenge, a news-driven stock market index prediction model based on TrellisNet and a sentiment attention mechanism (SA-TrellisNet) is proposed. A sentiment analysis model based on CNN and LSTM is presented to obtain the sentiment index of massive news crawled from authoritative financial websites. Furthermore, a sentiment attention mechanism is designed for data fusion of stock data and news sentiment index as the input of the simple and efficient TrellisNet network for model training and prediction. The performance of our model is systematically evaluated using seven major international stock market indices including S&P500, NYSE, DJI, NASDAQ, FTSE 100, Nikkei 225 and SSE, and comparative experiments demonstrate that SA-TrellisNet is competitive to the other state-of-the-art methods in predicting stock market indices. [ABSTRACT FROM AUTHOR]

The emergence of artificial intelligence (AI) and its related capabilities has led industries to rethink the existing practices of conventional supply chain management and data analysis. Machine learning (ML), Deep Learning (DL) and their unique ability to predict future data and classify data have led to important research in the supply chain (SC) domain, particularly in identifying and prioritizing supply chain risks. This paper proposes several DL methodologies to exploit the benefit of DL, particularly to identify whether any product will be delivered late due to any unforeseen reason in a complex SC system. Four different DL architectures (Simple-LSTM, Deep-LSTM, 1D-CNN, and TCN-1DSPCNN models) are proposed to extract features, while six variant classifiers: Softmax, random trees (RT), random forest (RF), K-nearest neighbor (KNN), artificial neural network (ANN), and support vector machine (SVM), were used to classify delay or non-delay information. By seamlessly capturing intricate temporal dependencies, these DL models enhance accuracy in robustly identifying supply chain late orders. Leveraging their hierarchical feature learning, these proposed DL models excel in recognizing subtle patterns and correlations, making them ideal for classifying late orders within the supply chain. Their parallel processing prowess facilitates real-time decision support, allowing organizations to address potential delays and allocate resources effectively and proactively. Five-fold cross-validation is presented to avoid over-fitting and to prove the efficiency of the proposed DL models. The total accuracies of the six ML classifiers are 74.03, 75.81, 93.35, 87.72, 93.59, and 95.10, respectively, while the maximum accuracies obtained from four proposed DL methodologies obtained an accuracy of 97.6, 98.63, 100, 100% respectively using the SVM classifier for predicting late orders based on five-fold cross-validation. • This paper investigates a few DL approaches to extract features of SC data. • Both RNN and CNN are applied in the same model. • An improved CNN model has been proposed for feature extraction. • An online dataset is employed to validate the proposed DL architectures. [ABSTRACT FROM AUTHOR]

Image colorization has a wide range of applications, but it remains a challenging task due to it is an inherently ill-posed problem with multi-modal uncertainty. The advancement of deep learning techniques has provided extensive avenues for addressing image colorization. However, current works mainly suffer from two problems: inaccurate colorization leading to biased color tones (e.g., cool or warm bias) and undersaturation of images. Existing Transformer-based methods can produce impressive results, but they often come with high training costs and may result in color overflow effects. In this paper, we propose a two-stage image colorization strategy based on a color codebook. Clustering methods in the three-dimensional CIE L a b color space is proposed to integrate brightness information so that the colors in the codebook can be lifelike. In the first stage, we treat the colorization task as a classification problem based on a color codebook, and a high-quality codebook is advantageous for enhancing color classification accuracy. In the second stage, different from the traditional Transformer-based method, a pyramid-type Transformer structure is used to extract rich image features to refine the colors, which can solve potential color bands, color errors and color overflow. In addition, the parameters and FLOPs are significantly smaller than other traditional Transformer-based methods. Extensive experiments demonstrate that our method outperforms state-of-the-art approaches. On the ImageNet validation set, the achieved values are 4.60, 25.23, 0.19, and 39.82 in terms of FID, PSNR, LPIPS, and CF, respectively. On the COCO-Stuff validation set, the achieved values are 5.62, 25.15, 0.19, and 36.25 in terms of FID, PSNR, LPIPS, and CF, respectively. The codes are available at https://github.com/Tanghui2000/Two-stage_Image_Colorization_via_Color_Codebook. • A novel two-stage image colorization approach has been proposed. • A color codebook is created in Lab 3-dimensional space by unsupervised clustering. • The colorization task is treated as a color classification task in the first stage. • A pyramid-type Transformer is adopted for refining the colors in the next stage. • Outperforms state-of-the-art methods in both quantitative and qualitative evaluation. [ABSTRACT FROM AUTHOR]

Automatic extraction of power transmission lines from remote sensing images is important for intelligent power inspection. However, this task faces many challenges, such as complex backgrounds and varying illumination, which make accurate segmentation difficult. Additionally, the severe class imbalance issue arising from the smaller pixel proportion of transmission lines compared to background makes segmentation even more challenging. To address these challenges, a multi-attention fusion network, named MAF-Net, is put forward to achieve accurate end-to-end segmentation of remote sensing images of transmission lines. Firstly, aimed at the inadequate process of local features, a multi-scale context fusion (MCF) block is put forward for effective feature aggregation of multi-scale features. Meanwhile, a dense block is also introduced for feature enhancement to acquire multi-scale contexts. Secondly, considering the pooling operation will bring the problem of information loss, a three-branch pooling (TP) block is proposed to take the best of the advantages of different pooling operations. In addition, the multi-scale input unit is also proposed to support deep layers for extracting the discriminative context information. Furthermore, to tackle the class imbalance issue, a multi-attention block (MA) is used to optimize the simple skip connections to fully aggregate spatial attention, pixel attention, and channel attention features, allowing this segmentation model to focus on the key areas containing power transmission lines. Comparison experiments and ablation experiments are carried out on the remote aerial sensing images for performance analysis. Experiments indicate that proposed MAF-Net could achieve a superior segmentation ability. • A multi-attention fusion network is proposed for accurate power line extraction. • A multi-scale context fusion block is proposed for effective feature learning. • A hybrid feature pooling block is to address the information loss issue. • Proposed model achieves a competitive performance on multiple public sets. [ABSTRACT FROM AUTHOR]

An intelligent recommendation system facilitates users by recommending items based on the user's preferences. Most of the existing recommender systems employ collaborative filtering which consider homogenous data only and does not exploit different knowledge sources. Such approaches encounter many challenges such as Cold Start problem, Sparsity, and Scalability. To overcome these setbacks, a novel deep learning & semantic fusion-based recommendation system named DLSF is proposed in this paper which collaborates multiple knowledge sources by employing three modules to generate top-N recommendations. The first module precludes sparsity by using latent representations and deep learning for understanding user-item interactions. The second module exploits semantic information provided to similar items by the users for dealing with cold start problem. And the third module works for new users and items which don't have any prior information. A comprehensive experimental evaluation is conducted over benchmark datasets. The proposed DLSF showed on average 16.02% and 16.71% more efficient over the baselines recommendation systems taken into account in terms of MAE and RMSE respectively. DLSF has found to have significant efficiency in terms of Precision, Recall and overall accuracy. [ABSTRACT FROM AUTHOR]

Android malware seriously affects the use of Android applications, and a growing number of Android malware developers are using adversarial attacks to evade detection by deep learning models. This work proposes an Android malware detection model based on the Android function call graph (FCG) and the denoising graph convolutional network (GCN) that is resistant to adversarial attacks. Methods are also proposed to simplify the FCG to reduce its size, and to construct vertex feature vectors. Because attackers may employ adversarial attack methods, the proposed model uses the subgraph network (SGN) to detect the underlying structural features of the FCG to discover the degree of the obfuscation attack. A denoising graph neural network (GNN) is designed, and the 1-Lipschitz-based neural network denoising method is applied to graph convolution. Moreover, the degree of denoising is adjusted according to the degree of obfuscation, which enhances the robustness of the model. The GCN performs the feature vector extraction of the FCG, and a multilayer perceptron (MLP) is used as the classifier. The results of experiments show that the F 1 value of the proposed Android malware detection method is higher than those of other malware detection models for different levels of obfuscation attacks, thus demonstrating its effectiveness against such attacks. [ABSTRACT FROM AUTHOR]

In wafer map defect pattern recognition, deep learning methods are predominantly used. These models autonomously learn features without explicit human intervention due to their black-box type network architectures. While preceding feature extraction and modality representation may seem neglected in deep learning for wafer map defect pattern recognition, we addressed this question by proposing an innovative approach. Our proposed method employs an antecedent feature learning module to create a more compact and informative hash modality representation from input wafer maps. The method exceeds in achieving high recall, crucial for identifying actual defective wafers as much as possible in semiconductor manufacturing, where defects can impact integrated circuit functionality and reliability. Smaller-sized hash modalities against larger wafer maps allow the same model to speed up the process and require fewer computational resources. [ABSTRACT FROM AUTHOR]

Multiple Instance Learning (MIL) is a learning paradigm, where training instances are arranged in sets, called bags, and only bag-level labels are available during training. This learning paradigm has been successfully applied in various real-world scenarios, including medical image analysis, object detection, image classification, drug activity prediction, and many others. This survey paper presents a comprehensive analysis of MIL, highlighting its significance, recent advancements, methodologies, applications, and evolving trends across diverse domains. The survey begins by explaining the core principles that form the basis of MIL and how it differs from traditional learning approaches. This sets the foundation for comprehending the distinct challenges and techniques of solving MIL problems. Next, we discuss how supervised learning algorithms are tailored to support MIL and combine this discussion with a review of seminal MIL algorithms as well as the latest innovations that incorporate neural networks, deep learning architectures, and attention techniques. This comprehensive analysis helps to understand the strengths, limitations, and adaptability of these methods across diverse data modalities, complexities, and applications. In summary, this survey paper provides an essential resource for researchers, practitioners, and enthusiasts seeking a comprehensive understanding of Multiple Instance Learning. It covers foundational concepts, traditional methods, recent advancements, and future directions. By providing a holistic view of MIL's dynamic landscape, this paper aims to inspire further innovation and exploration in this ever-evolving field. • A survey on the current state of the Multiple Instance Learning. • We provide applications of Multiple instance learning in various domains. • We discuss how existing supervised learning algorithms are modified to support MIL. • Publicly available datasets and open research challenges in MIL are discussed. [ABSTRACT FROM AUTHOR]

The assessment of grape ripeness is an extremely important factor in winemaking and has a direct impact on wine quality. This process is usually carried out with a traditional laboratory analysis, a costly procedure that destroys the grapes selected for analysis. Consequently, the research in precision viticulture has shifted focus to the development of digital processes that are fast and non-intrusive. In this context, the use of hyperspectral imaging paired with prediction models for the estimation of oenological parameters has gained wide recognition. The major drawback of these solutions is the extreme variability presented by the data, aligned with a small number of samples for training, derived from the high cost of acquiring new samples infield. Achieving a satisfactory generalization capacity while working on small data sets with such high variability is a serious challenge, and in this work we aim to provide a pipeline on how to properly build validation and test sets that allow for a correct evaluation of performance, avoiding common misconceptions such as using the R 2 metric for model selection or creating models based on biased data sets. Additionally, we implement and evaluate different architectures, namely Residual Networks, InceptionTime and MiniRocket, to showcase that deep learning techniques can accurately predict sugar content from different vintages and varieties of wine grape berries, maintaining a strong generalization capacity even in a setting of high variability and small number of samples. Finally, our results also suggest that adding more relevant features to better characterize the data might be enough for the networks to adjust and produce accurate estimates of sugar content, which would eliminate the necessity to capture new samples on a yearly basis. • Deep learning models applied in sugar content estimation from hyperspectral images. • Directions for building sets that allow the assessment of generalization capacity. • Deeper networks get a good generalization capacity on unseen samples of wine grapes. • MiniRocket presents itself as a fast alternative to high-parameterized models. • Results show that a broader set of features might be key to achieve a scalable model. [ABSTRACT FROM AUTHOR]

Deep learning systems, particularly those used in image classification, are threatened by Adversarial Examples. In contrast, Adversarial Examples do not affect the mammalian visual system. We undertake a comparative analysis of the traditional image multi-classification models and human cognitive frameworks, namely ACT-R and QN-MHP, and find that the One-hot encoded output structure lacks anatomical support. Furthermore, the CLIP model, which uses natural language supervision, closely resembles the human visual cognition process. There is structural information between category labels, and various label errors are not equivalent in most cases. One-hot encoding disregards these distinctions, exacerbating Adversarial Examples' detrimental impact. We introduce a new direction for the adversarial defense that replaces One-hot encoding with natural language encoding or other encodings that preserve structural information between labels. Experiments and Lipschitz continuity analysis show that this approach can enhance the robustness of the model against Adversarial Samples, especially in scenarios such as visual symbol recognition. • We improve the concept of adversarial robustness by assessing label error degrees. • We find encoding with One-hot can exacerbate the harmful of Adversarial Examples. • We preserve category structure information to enhance adversarial robustness. [ABSTRACT FROM AUTHOR]

• Propose a multi-task deep learning model to predict real-time and sparse travel demand. • Propose an algorithm converting repeated calls into travel demand. • Employ zero-inflated loss function to handle data sparsity issue. • Apply the proposed model to demand responsive transit data in South Korea. • Multi-task structure with zero-inflated loss can mitigate the data sparsity problem. Demand Responsive Transport (DRT) is a flexible transportation service that adjusts its routes and schedules in real time based on user requests. Real-time demand forecasting has become a timely issue since the resolution of transportation planning has increased to provide flexible services to users. Although many previous studies have been conducted to design the demand forecasting model using the mobility service historical data, there are more considerations when predicting real-time demand for on-demand mobility services. Therefore, this study aims to develop a demand forecasting model for DRT services addressing two issues. First, repeated calls for service and abandoned requests are converted into the unit of demand to capture served and potential demand. Second, given the sparsity of DRT historical data, a multi-task deep learning model is proposed to predict both the probability and volume of demand simultaneously to mitigate the data sparsity problem. A zero-inflated loss function was introduced to consider the skewed demand distribution. To attain our goals, the historical dataset of the I-MOD service trial in Yeongjongdo, Inchon, Korea was used. The performance of the proposed model is compared against several baseline models, demonstrating its superiority in forecasting demand for mobility services. The results show that a multi-task structure can alleviate the data sparsity problem by excluding where demand is not generated from the forecasting problem. The proposed models can provide insights into real-time demand forecasting with sparse demand. [ABSTRACT FROM AUTHOR]

Compressive sensing (CS) is a kind of hardware-friendly technology in effective image reconstruction. Most of the existing CS reconstruction methods can be classified into model-driven iterative optimization methods and data-driven neural network methods. These two types of methods have their own problems with computational complexity and theoretical interpretability. Motivated by the deep unfolding theory, we develop the model-driven Extragradient-based Iterative Denoising Network, dubbed EIDNet, which is based on the denoising model of the iterative shrinkage threshold algorithm (ISTA). By unfolding the iterative denoising model into the deep learning network, EIDNet addresses the interpretability problem of neural network methods in CS reconstruction. Each denoising iteration of the optimization algorithm is mapped into one layer of the reconstruction network in EIDNet. The extragradient method is adopted to accelerate network convergence. In addition, we introduce a learning-based sampling and initialization network for adaptive sampling and initialization. Extensive experiments show that the proposed EIDNet can obtain better image CS reconstruction performance than other state-of-the-art CS methods while maintaining satisfactory reconstruction efficiency. • A model-driven network is developed for image reconstruction. • The learning-based sampling and initialization network is introduced. • The extragradient method is adopted to accelerate network convergence. • The method is experimented on natural images and magnetic resonance images. [ABSTRACT FROM AUTHOR]

In the context of early childhood education, students need to acquire fundamental writing skills for their lifelong development. Public schools, especially in low- and middle-income countries, should monitor individual student progress to mitigate the detrimental effects of socioeconomic vulnerabilities in education. Furthermore, the volume of students often overwhelms teachers responsible for assessing handwriting texts and providing feedback. This article conducts a Systematic Literature Review (SLR) focusing on solutions for automatically evaluating students' handwriting, discussing their performance, future research directions, and areas needing further investigation. We used a mixed-methods approach to conduct our SLR, encompassing a search across four databases (ACM Digital Library, IEEE Xplore, ScienceDirect, and SpringerLink) and employed the snowballing technique. We used the initial set of papers from the database search as the foundation for the subsequent snowballing search. Findings revealed that the studies introduced computational techniques, examined or enhanced existing methods, and developed assessment tools. These solutions predominantly employed techniques such as artificial neural networks and used available datasets comprising handwritten images, answers, or student essays. Only some studies provide low-cost solutions for automatically assessing the writing abilities of underserved public school students. • We reviewed 491 papers, extracting data from 22 focusing on handwriting assessment. • We applied a mixed method using database search and the snowballing technique. • Studies frequently adopt deep learning methods to tackle the handwriting recognition problem. • Studies employ various methods to automatically assess text production, such as ANN, latent semantic analysis, content vector analysis, and CNN. • The mean accuracy for handwriting assessment was notably high at 93.64%, suggesting that the models in this group exhibited good performance. [ABSTRACT FROM AUTHOR]

This paper addresses the crucial task of power line detection and localization in electrical infrastructure inspection using Unmanned Aerial Vehicles (UAVs) from weak supervision, polyline annotations. We first identify several limitations in the state-of-the-art approach LSNet. In particular, the inability of LSNet to detect line-crossings and lines in close proximity. To overcome these limitations, we propose LSNetv2, which enhances LSNet with multi-line segment detection capability facilitated via a bipartite matching loss. Additionally, we update LSNet's regression loss in order to stabilize training by reducing the interdependence between predicted coordinates. Finally, LSNetv2 makes use of an increased receptive field to extract global information, improving overall detection performance. Through extensive evaluations on various power line detection datasets, LSNetv2 demonstrates superior performance and robustness. On the public datasets PLDU, PLDM and TTPLA, it achieved F β scores of 0.857, 0.875, and 0.671, respectively, while using only modified weak polyline annotation, establishing itself as an effective and efficient solution for power line detection in UAV-based electrical infrastructure inspections. • Visual power line detector using (weak) polyline annotation. • Bipartite matching loss facilitates accurate detection of overlapping lines. • Improved feature extractor and regression loss compared to predecessor LSNet. • Empirical evaluation demonstrates LSNetv2's benefits. [ABSTRACT FROM AUTHOR]

Adversarial attacks can mislead deep learning models to make false predictions by implanting small perturbations to the original input that are imperceptible to the human eye, which poses a huge security threat to computer vision systems based on deep learning. Physical adversarial attacks, which is more realistic, as the perturbation is introduced to the input before it is captured and converted to a image inside the vision system, when compared to digital adversarial attacks. In this paper, we focus on physical adversarial attacks and further classify them into invasive and non-invasive. Optical-based physical adversarial attack techniques (e.g. using light irradiation) belong to the non-invasive category. The perturbations can be easily ignored by humans as the perturbations are very similar to the effects generated by a natural environment in the real world. With high invisibility and executability, optical-based physical adversarial attacks can pose a significant or even lethal threat to real systems. This paper focuses on optical-based physical adversarial attack techniques for computer vision systems, with emphasis on the introduction and discussion of optical-based physical adversarial attack techniques. [ABSTRACT FROM AUTHOR]

This paper proposes physics-informed meta-learning-based surrogate modeling (PI-MLSM), a novel approach that combines meta-learning and physics-informed deep learning to train surrogate models with limited labeled data. PI-MLSM consists of two stages: meta-learning and physics-informed task adaptation. The proposed approach is demonstrated to outperform other methods in four numerical examples while reducing errors in prediction and reliability analysis, exhibiting robustness, and requiring less labeled data during optimization. Moreover, compared to other approaches, the proposed approach exhibits better performance in solving out-of-distribution tasks. Although this paper acknowledges certain limitations and challenges, such as the subjective nature of physical information, it highlights the key contributions of PI-MLSM, including its effectiveness in solving a wide range of tasks and its ability in handling situations wherein physical laws are not explicitly known. Overall, PI-MLSM demonstrates potential as a powerful and versatile approach for surrogate modeling. • A proposed approach uses meta-learning and PIDL to train surrogates with small data. • A model-agnostic meta-learning model is introduced to learn surrogate model weights. • A PIDL approach is introduced to adapt to target tasks with meta-learned weights. • Our approach outperformed other methods in numerical examples given limited data. • Proposed approach showed robustness in solving out-of-distribution tasks. [ABSTRACT FROM AUTHOR]

As a vast number of process data composed of three-dimensional (3D) computer-aided design (CAD) model and its associated machining process are generated each year, data-driven machining process planning is becoming an effective strategy for engineers to generate the process scheme of a part with less time and lower cost. However, variety of knowledge in these data have not been analyzed, learned and used effectively, and thus the numerical control (NC) process scheme of the part still relies on the skilled engineers. In this work, we propose an effective NC machining process planning method via integrating grammar knowledge with deep learning. First, the process knowledge And-Or graph (PK-AOG) is defined through the use of a context-free grammar for its compositional properties, and it forms the solution space for the working step sequence of a part, which is a parsing graph of PK-AOG essentially. Then, the implicit mapping mode between each machining feature and its feature process label are learned through the use of deep learning method based on attention mechanism for the structured process data, and the probability distributions of different feature process labels are calculated. Finally, taking PK-AOG as guidance, the optimal process scheme for a part, which are composed of working step sequence, its compatible feature processes, and the machining process of each machining feature, is jointly searched via integrating ant colony optimization (ACO) and genetic algorithm (GA) effectively, and thus a process scheme of a part with correct logics and semantics is generated. The proposed approach is verified by the experiments on the developed prototype system based on computer aided three-dimensional interactive application (CATIA) software. [ABSTRACT FROM AUTHOR]

Automatic segmentation of abdominal organs in computed tomography (CT) images can support radiation therapy and image-guided surgery workflows. Development of such automatic solutions remains challenging mainly owing to complex organ interactions and blurry boundaries in CT images. To address these issues, we focus on effective spatial context modeling and explicit edge segmentation priors. Accordingly, we propose a 3D network with four main components trained end-to-end including shared encoder, edge detector, decoder with edge skip-connections (ESCs) and recurrent feature propagation head (RFP-Head). To capture wide-range spatial dependencies, the RFP-Head propagates and harvests local features through directed acyclic graphs (DAGs) formulated with recurrent connections in an efficient slice-wise manner, with regard to spatial arrangement of image units. To leverage edge information, the edge detector learns edge prior knowledge specifically tuned for semantic segmentation by exploiting intermediate features from the encoder with the edge supervision. The ESCs then aggregate the edge knowledge with multi-level decoder features to learn a hierarchy of discriminative features explicitly modeling complementarity between organs' interiors and edges for segmentation. We conduct extensive experiments on challenging abdominal CT datasets with eight annotated organs. Experimental results show that the proposed network outperforms several state-of-the-art models, especially for the segmentation of small and complicated structures (gallbladder, esophagus, stomach, pancreas and duodenum). • Proposes an end-to-end 3D network for the challenging abdominal organ segmentation. • Focuses on effective spatial context modeling and explicit edge segmentation priors. • Incorporates wide-range contextual dependencies via directed graphs. • Aggregates the learned edge priors with multi-level decoder features. • Promising results in comparison to several cutting-edge models. [ABSTRACT FROM AUTHOR]

• Benchmarking for the first-time, deep learning models for depression detection on the CLEF2017 dataset. • Proposing an attention-based CNN-BiLSTM model that outperforms existing deep learning models for depression detection. • Comprehensive evaluation using statistical metrics & Accuracy, F1, Precision, AUC-ROC, MCC & significance of AUC-ROC, MCC. Depression has long been described as a common mental health disorder and a disease with a set of diagnostic criteria that influences the affected individuals' feelings and behavior. The prevalence of Internet use has augmented people's openness to share their experiences and struggles, including mental health disorders on social media thus researchers have tried developing classification models for depression detection using various machine learning and deep learning techniques. In this research, we propose a deep learning architecture with an attention mechanism on CNN-BiLSTM (CBA) and provide a comparative analysis to benchmark well-known deep learning models using the public dataset namely CLEF2017. We found that along with F1 score, precision and recall it is also vital to consider the Area under the curve - Receiver operating characteristic curve (AUC-ROC) and Mathews Correlation Coefficient (MCC) metrics for evaluating depression classification models since the MCC considers all the four values of a confusion matrix. Based on our experiments, the CBA model outperforms the existing state of the art model with an overall accuracy of 96.71% and scores of 0.85 and 0.77 for AUC-ROC and MCC, respectively. [ABSTRACT FROM AUTHOR]

In vehicular accidents, the swift and accurate identification of car crashes is paramount, as it serves as the linchpin for prompt responses from emergency services and search-and-rescue operations. This study introduces an innovative multimodal car crash detection system that capitalizes on audio-visual data sourced from dashboard cameras, thus significantly enhancing the precision of automobile collision detection. In contrast to car crash detection systems relying on single-modal inputs, this research employs advanced multimodal methodologies with a neural network architecture that integrates Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) for car crash detection. This system concurrently leverages audio spectrogram and audio feature data, video RGB, video optical flow, and video differential information, enabling collision recognition through a neural network system, resulting in markedly improved collision detection accuracy. A comprehensive comparative analysis is conducted, benchmarking our system against the most cutting-edge existing car crash detection systems. The system employs a unique fusion approach, allowing it to apply its inference to audio-visual information of varying durations compared to the training set. The empirical findings unequivocally affirm our proposed solution's substantial enhancement in car crash detection. • Introduction of a Novel Audio-Visual Fusion Approach. • Multimodal Processing-Based Vehicle Collision Detection System. • Utilization of RGB, Optical Flow, and Difference Processing for Video Data. • Inference Applicable to Testing Datasets of Varying Lengths. [ABSTRACT FROM AUTHOR]

It is a crucial means for humans to perceive geomorphic features and landscape architectures by classifying ground objects in hyperspectral images (HSIs). Currently, the exponential development of neural networks has provided a powerful support for the accurate HSI classification. However, existing neural network-based methods usually rely solely on the data to drive the classification model, lacking attention to valuable land-cover distribution knowledge in HSIs. In view of this, to utilize hyperspectral data and distribution knowledge simultaneously, a data and knowledge-driven deep multiview fusion network based on diffusion model (DKDMN) is proposed in this paper. DKDMN extracts knowledge from unlabeled data in HSIs through a diffusion model-based knowledge learning framework (DMKLF), and combines raw hyperspectral data with the acquired knowledge through a designed deep multiview network architecture (DMNA) to mine complicated land-cover distribution information and reflect sample relationships. First, the proposed DMKLF utilizes the data distribution reconstructed by the diffusion model as a knowledge source for one view to enhance the network cross-sample awareness ability. On the other hand, the original HSI patches are considered a data source for another view, which co-drives DMNA with the unsupervised diffusion knowledge extracted by DMKLF to perform effective feature extraction. Second, taking into account the characteristics of each view and the feature similarity between these two views, a joint loss function specifically for DMNA is suggested to minimize the difference between the model predictions and the real labels. Finally, a multi-backbone integration classification framework (MBICF) is designed by deeply fusing three vision architectures to capture multi-scale spectral features and local–global features, thereby achieving pixel-wise classification effectively. Experimental results on four publicly available HSI datasets demonstrate that the proposed DKDMN achieves competitive classification accuracy compared with other state-of-the-art methods. For instance, the proposed DKDMN achieves an overall accuracy improvement of 1.62% and 2.18% on the Indian Pines and Salinas Valley datasets, respectively, compared to the multiple vision architecture-based hybrid network (MVAHN). The related code will be released at https://github.com/ZJier/DKDMN. • Designing a data and knowledge-driven HSI classification method. • Learning unsupervised spectral spatial distribution knowledge. • Combining data source and knowledge source by deep multiview network architecture. • Realizing HSI classification by multi-backbone integration classification framework. [ABSTRACT FROM AUTHOR]

The prevalence of security vulnerabilities in Internet of Things (IoT) applications poses a serious threat to enterprise systems, necessitating sophisticated and reliable defense solutions to counter emerging and evolving threats. For the Industrial Internet of Things (IIoT), stakeholders require trustworthy and sustainable systems that can prevent the loss of human life during critical operations. The impact of multi-variant persistent and sophisticated bot attacks on connected IIoTs is potentially catastrophic, and their detection presents a highly complex and critical challenge. Therefore, there is a pressing need for efficient and timely detection of IIoT botnet attacks. This research paper proposes a robust deep learning model named AttackNet for the detection and classification of different botnet attacks in IIoT based on adaptive based CNN-GRU model. The model is extensively evaluated using the latest dataset and standard performance evaluation metrics, demonstrating its capacity to protect IIoT networks against sophisticated cyber-attacks with a testing accuracy of 99.75%, a loss of 0.0063, precision and recall score of 99.75% and 99.74% respectively. Our proposed model demonstrates superior accuracy, particularly within the N_BaIoT dataset. It achieves an outstanding accuracy of 99.75% across ten classes, surpassing state-of-the-art techniques by a substantial margin ranging from 3.2% to 16.07%. Moreover, the proposed model outperforms state-of-the-art anomaly detection systems in IIoT based on a real-time IoT device dataset in terms of detecting and classifying botnet attacks accurately. [ABSTRACT FROM AUTHOR]

In artificial intelligence applications, advanced computational models, such as deep learning, are employed to achieve high accuracy, often requiring the execution of numerous operations. Conversely, lightweight computational models are typically more resource-efficient, making them suitable for various devices, including smartphones, tablets, and wearable technology. This paper presents an ultra-low-computation solution for interpreting sign languages to assist deaf and hard-of-hearing individuals without needing specialized hardware or significant computational resources. The proposed approach initially performs data abstraction on the input data. During this process, the image is systematically scanned from various perspectives, and the collected information is then encoded into a one-dimensional vector. Subsequently, the abstracted information undergoes processing through a Fully Connected Neural Network (FCN), resulting in highly accurate output. We also introduced two abstraction methods, namely Opaque and Glass, inspired by the interaction of light with different types of objects. The proposed abstractions facilitate the comprehension of the hand gesture's outer boundary as well as its row-wise and column-wise density of pixels. Our experiments on three datasets confirm the efficiency of the proposed method, achieving an accuracy of 99.4% in recognizing American Sign Language, 99.96% accuracy in recognizing Indian Sign Language, and 99.95% accuracy in recognizing Bangla Sign Language. Notably, the model size and the number of MAC operations are significantly smaller than state-of-the-art computational models trained on the same datasets. • We introduced an ultra-low-computation method for interpreting sign languages. • We developed a new encoding for the abstraction of hand gesture images. • Our approach significantly reduces both model size and MAC operations. • We achieved high accuracy in understanding American, Indian, and Bangla SLs. [ABSTRACT FROM AUTHOR]

Lameness is one of the major health problems on dairy farms, which seriously affects dairy cow welfare and increases the risk of premature culling. Accuracy lameness detection ensures timely treatment of hooves and improves the level of health management on dairy farms. Most of the existing lameness detection methods detect one dairy cow, and it is difficult to detect lameness in multiple dairy cows at the same time. In this paper, a lameness detection system based on instance segmentation is constructed to extract the lameness features of multiple dairy cows and automatically detect lameness. First, an improved SOLOv2 network is designed to reduce the semantic gap between low-level and high-level features and improve the precision of dairy cow segmentation. Second, individual matching of dairy cows in different video frames is performed using the Hungarian algorithm. Third, the Canny algorithm is used to extract the neck and back contour features and key gait features of dairy cows. Finally, a deep learning model is constructed, and multiple lameness features are fused to detect the lameness of dairy cows. To evaluate the performance of the constructed system, 172 videos were randomly selected from 246 videos as training videos, and the remaining 74 videos were selected as the test videos. The lameness detection accuracy of the constructed system was 98.65%. The experimental results showed that the constructed system can extract the lameness features of multiple dairy cows at the same time and accurately detect the lameness of dairy cows. [ABSTRACT FROM AUTHOR]

Federated Learning is a collaborative machine learning paradigm that allows training models on decentralized data while preserving data privacy. It has gained significant attention due to its potential applications in various domains. However, the issue of protecting model copyright in the Federated Learning setting has become a critical concern. In this paper, we propose a novel watermarking framework called FedCRMW (Federal Learning Compression-Resistance Model Watermark) to address the challenge of model copyright protection in Federated Learning. FedCRMW embeds unique watermarks into client-contributed models, ensuring ownership, integrity, and authenticity. The framework leverages client-specific identifiers and exclusive logos to construct trigger sets for watermark embedding, enhancing security and traceability. One of the key advantages of FedCRMW is its optimization for the common data compression challenge in the Federated Learning scenario. By utilizing compressed data inputs for copyright verification, we achieve an efficient watermark validation process and reduce communication and storage overheads. Experimental results demonstrate the effectiveness of FedCRMW in terms of watermark success rate, imperceptibility, robustness against attacks, and resistance to model compression and pruning. Compared to existing watermarking methods, FedCRMW exhibits superior performance in the Federated Learning context. • FedCRMW: Robust watermarking for Federated Learning models. • Novel trigger dataset construction scheme for watermarking. • Enhanced robustness with feature-consistent training. [ABSTRACT FROM AUTHOR]

The growing demand for cloud computing and storage necessitates the expansion of Data Centers, thereby increasing energy consumption and environmental footprint. The data-driven dynamic thermal control presents a promising solution to mitigate this issue, as it can offer improved thermal control strategies and address the intricate and stochastic nature of the thermal environment. Nonetheless, existing solutions typically exhibit deficiencies in terms of reliability and practicability. To this end, this paper proposes a feasible uncertainty-aware online learning method with reliable dynamics models and controllers. Specifically, Deep Learning models with the capacity for uncertainty quantification are employed to account for the imperfections in pre-trained models, hence providing reliable predictions. The Cross-Entropy Method and Monte Carlo trajectory sampling solve optimization problems while considering uncertainties. It is mathematically analyzed that the uncertainty quantified by the system dynamics model can be captured with sufficient trajectory samples, rendering a reliable controller. A practical framework is proposed and tested on a widely used Data Center model to mitigate the insufficient capacity of Deep Learning models to quantify uncertainties. The results demonstrate its superior performance during the initial training stage, achieving a 66% reduction in violations and 3.76% power savings compared to the default controller. Continuous active online learning enhances its adaptability, resulting in an 80% reduction in violations and a 6% energy saving. The thermal control strategies are thus improved by the proposed new solutions and insights. [ABSTRACT FROM AUTHOR]

Breast cancer, with its high mortality, faces diagnostic challenges due to variability in mammography quality and breast densities, leading to inconsistencies in radiological interpretations. Computer-aided diagnostic (CAD) systems, while helpful, struggle with accurately interpreting lesion characteristics such as morphology, density, and size. To address this, our study developed advanced deep-learning algorithms to improve the detection, localization, risk assessment, and classification of breast lesions, aiming to reduce false positives on human intervention and tackle slow convergence rates. Key innovations of the approach include preprocessing techniques, advanced filtering, and data augmentation strategies to optimize model performance, mitigating over- and under-fitting concerns. A significant development is the Chaotic Leader Selective Filler Swarm Optimization (cLSFSO) method, which effectively detects breast-dense lesions by extracting textural and statistical features. Additionally, the study adapted deep learning models like modified VGGNet and SE-ResNet152 through transfer learning, significantly enhancing their capability to distinguish between normal and suspicious mammography regions. The study also introduces hybrid deep neural network-based approaches, including CNN+LSTM and CNN+SVM, for diagnosing and grading cancerous polyps from the pre-segmented ROIs. Besides, the transfer learning paradigm is employed to boost the efficacy in classifying breast masses and reducing computing time by modifying the final layer of the proposed pre-trained models. The integration of Grad-CAM techniques further refines our analysis, leading to more accurate assessments and improved diagnosis of breast anomalies. Evaluated using benchmark and private datasets, our algorithms demonstrated a sensitivity of 0.99 and an overall AUC of 0.99, indicating significant improvements in mammogram analysis. These advancements aid radiologists, potentially improving patient outcomes and contributing to medical imaging and AI in healthcare. • To mitigate image impairments and boost contrast using new enhancement schemes. • To extract ROIs from preprocessed images based on density by cLSFSO Algorithm. • To extract intricate features from mammograms using mVGGNet and SEResNet152 models. • To develop robust DCNN-based hybrid algorithms for precise breast lesion grading. • To boost breast cancer detection and grading using GradCam techniques. [ABSTRACT FROM AUTHOR]

Counting repetitive actions is important in work and daily life. Automated counting using deep learning provides a more efficient, accurate alternative to manual counting, which is tedious and error-prone Deep-learning models have been proposed to automatically count repetitive actions in video content. However, for these models to be applied to realistic scenes, high-quality performance and generalization to multiple environments, particularly for long videos, are essential. To address these challenges, we propose a new model, ME-RAC, which includes the multipath 3D-Conv encoder module, and we also propose a temporal-sequence random-combination data augmentation to improve counting performance and prevent model over-fitting during training. Additionally, we propose the temporal-sequence-decision (TSD) framework system to realize long repetitive-action counting in complex realistic scenes. We conducted experiments to validate that our proposed methods perform better than comparable methods and our TSD framework achieved unique performance in long repetitive-action-counting tasks. • ME-RAC model can enhance the performance of repetitive action counting. • TSRC data augmentation method can mitigate overfitting issue during training. • TSD framework effectively implements action counting in long-time video. [ABSTRACT FROM AUTHOR]

With the large chunks of social media data being created daily and the parallel rise of realistic multimedia tampering methods, detecting and localising tampering in images and videos has become essential. This survey focusses on approaches for tampering detection in multimedia data using deep learning models. Specifically, it presents a detailed analysis of publicly available benchmark datasets for malicious manipulation detection. It also offers a comprehensive list of tampering clues and commonly used deep learning architectures. Next, it discusses the current state-of-the-art tampering detection methods, categorizing them into meaningful types such as deepfake detection methods, splice tampering detection methods, copy-move tampering detection methods, etc. and discussing their strengths and weaknesses. Top results achieved on benchmark datasets, comparison of deep learning approaches against traditional methods and critical insights from the recent tampering detection methods are also discussed. Lastly, the research gaps, future direction and conclusion are discussed to provide an in-depth understanding of the tampering detection research arena. [ABSTRACT FROM AUTHOR]

Adversarial sample generation problem is a hot issue in the security field of deep learning. Evolutionary algorithm has been widely used to solve this problem in recent years because of its good global search ability. However, existing methods still suffer from the "curse of dimensionality" when attacking high-resolution images. In this paper, a two-stage frequency domain generation algorithm of black-box adversarial samples based on differential evolution is proposed. In the first stage, a representative image-guided differential evolution method is proposed to quickly generate a universal adversarial perturbation with a high attack success rate in the frequency-domain. In the second stage, a space reduction strategy based on frequency-domain pixel blocks is designed to reduce the search space and alleviate the problem of "curse of dimensionality". In addition, a new space–frequency interaction sensitivity measure is introduced to evaluate the similarity between the adversarial samples and the original images. The adversarial perturbations obtained by the measure are more in line with the subjective perception of the human eye. Finally, compared with several typical black-box adversarial sample generation algorithms, experimental results show that the proposed algorithm can achieve higher attack success rate with less prediction times. • Proposing a two-stage evolutionary generation framework of adversarial samples. • Developing a space–frequency interaction sensitivity measure. • Presenting a representative image-guided differential evolutionary algorithm. • Designing an individual adversarial perturbation generation algorithm. [ABSTRACT FROM AUTHOR]

Stock price forecasting is a challenging task because financial time series are primarily nonlinear, noisy, and disordered systems that are complicated to forecast. Deep learning models show promise in this domain along with natural language processing, to extract relevant features from text data and map them to numerical representations. This study aims to forecast stock prices using text analysis and deep learning approaches and explain the models using explainable AI. We construct a World Halal Tourism Composite Sentiment Index (WHTCSI) using text analysis to forecast halal tourism stock price. The results suggest that Convolutional Neural Networks (CNN) outperform all other models. The results are robust when considering country-level data. In addition, model explanations show that the index contributes 35.55% to the forecasting model, indicating irrational investment activity and herding behavior in the halal tourism industry. The study's findings have significant implications for investors, analysts, and portfolio managers in making investment decisions. [ABSTRACT FROM AUTHOR]

Mesoscale Eddy (ME) is a widely recognized and significant oceanic phenomenon characterized by extensive energy exchange. Accurate predictions of the future geospatial distribution of ME are crucial for maritime activities. Deep learning (DL) methods for ME prediction have consistently outperformed classical mathematical models and numerical modeling approaches. However, current DL methods based on two-dimensional gridded data suffer from error accumulation issues due to their indirect prediction approach, which involves first predicting the future environmental field and then identifying the ME patterns within it. Additionally, these models ignore the potential influence of prior knowledge related to ME on prediction outcomes. To address these issues, this paper proposes a direct prediction DL network called ES-ConvGRU, which incorporates historical prior statistics. First, we conduct an analysis of ME data from the Northwest Pacific region, extracting prior statistical knowledge such as interannual variations and seasonal characteristics. Second, mining patterns for the spatiotemporal features of both ME sequence and sea surface environmental field sequence are established. Furthermore, a nonlinear approach for mapping prior statistical knowledge and a multi-step autoregressive prediction model are designed, facilitating the integration of prior knowledge with multidimensional spatiotemporal features for ME prediction. Finally, the performance and effectiveness of ES-ConvGRU are evaluated. It achieves an average pixel accuracy of 93.21%, a mean intersection over union of 81.48%, and a frequency-weighted intersection over union of 87.55% for the 7-day distribution prediction. The results show that ES-ConvGRU exhibits notable advantages compared to the existing indirect and direct prediction methods. • A direct predictive approach for mesoscale eddy distribution prediction is developed. • Prior seasonal statistics is considered for prediction. • An end-to-end deep prediction network architecture, ES-ConvGRU, is proposed. • The results show that ES-ConvGRU performs better than other predictive methods. [ABSTRACT FROM AUTHOR]

Images captured under low-light conditions usually co-exist with low light and blur degradation. Most existing cascade and joint enhancement methods may provide undesirable results, suffering from severe artifacts, deteriorating blur, and unclear details. In this paper, we propose a novel network with structural priors, including high-frequency and edge, to enable effective image representation learning for joint low-light enhancement and deblurring. Specifically, we employ a Transformer backbone to explore the global clues of the image. To compensate for the inadequate local detail optimization, we propose a multi-patch perception pyramid block that models the correlation between different size patches and ambiguity, and identifies non-uniform deblurring spatial features, facilitating the reconstruction of potential high-frequency and edge information. Furthermore, a prior-guided reconstruction block based on the parallel attention mechanism is present to adaptively correct global image with statistical features, which helps guide the model to refine sharp texture and structure. Extensive experiments performed on simulated and real-world datasets demonstrate the efficacy of our proposed method in restoring low-light blurry and low-light images with increased visual perception compared to state-of-the-art methods. • Multi-structural priors are employed to joint low-light enhancement and deblurring. • A multi-patch perception pyramid block to remove non-uniform is proposed. • Prior-guided reconstruction block to adaptively correct global features is present. • The proposed method is extensively evaluated and attains superior performance. [ABSTRACT FROM AUTHOR]

The task of multimodal image fusion aims to preserve the respective advantages of each modality, such as the detailed texture information from visible light images and the salient target information from infrared images. However, images in real environments are often not perfect high-quality images but are affected by various degradation factors such as noise, blur, and glare. Existing unsupervised multimodal image fusion algorithms use undegraded source images as input to fusion networks and generate high-quality fusion images through complex fusion strategies. This single linear learning approach from high-quality to high-quality cannot be applied to the fusion task of degraded multimodal images in real environments. To address this issue, this paper proposes an end-to-end multimodal image fusion network based on a high-order local degradation model (MFHOD). Firstly, inspired by the idea of probabilistic degradation, we propose a high-order local random degradation model (HODM), which inputs the source multimodal images into the degradation model to obtain degraded images before feeding them into the network. Secondly, we design a simple and efficient dual-branch feature extraction encoder to extract deep features from images. Then, from the perspectives of image pixels, brightness, and gradients, we propose an improved composite loss composed of multiple loss functions to constrain network training. Finally, we propose a L 2 -norm fusion strategy for preserving brightness information in low-light nighttime images. Our MFHOD demonstrates good performance on infrared and visible light image datasets as well as medical image datasets. Experimental results show that MFHOD can effectively suppress the effects of glare, noise, and smoke in adverse environments, and also improve the quality of fusion images in low-light and nighttime environments. • Linear fusion methods cannot fully train the feature extraction of neural networks. • Two-stage high-order degradation models significantly boost network nonlinearity. • End-to-end network simplifies multimodal image fusion process. • MFHOD boosts brightness and contrast, suppressing glare, noise, and smoke effects. [ABSTRACT FROM AUTHOR]

Cognitive load theory suggests that overloading of working memory may negatively affect the performance of human in cognitively demanding tasks. Evaluation of cognitive load is a difficult task; it is often assessed through feedback and evaluation from experts. Cognitive load classification based on Functional Near-InfraRed Spectroscopy (fNIRS) is now one of the key research areas in recent years, due to its resistance of artefacts, cost-effectiveness, and portability. To make fNIRS more practical in various applications, it is necessary to develop robust algorithms that can automatically classify fNIRS signals and less reliant on trained signals. Many of the analytical tools used in cognitive sciences have used Deep Learning (DL) modalities to uncover relevant information for mental workload classification. This review investigates the research questions on the design and overall effectiveness of DL as well as its key characteristics. We have identified 45 studies published between 2011 and 2023, that specifically proposed Machine Learning (ML) models for classifying cognitive load using data obtained from fNIRS devices. Those studies were analyzed based on type of feature selection methods, input, and DL model architectures. Most of the existing cognitive load studies are based on ML algorithms, which follow signal filtration and hand-crafted features. It is observed that hybrid DL architectures that integrate convolution and LSTM operators performed significantly better in comparison with other models. However, DL models especially hybrid models have not been extensively investigated for the classification of cognitive load captured by fNIRS devices. The current trends and challenges are highlighted to provide directions for the development of DL models pertaining to fNIRS research. [ABSTRACT FROM AUTHOR]

Extensive research has been conducted on image steganography and watermarking algorithms, owing to their crucial rules in secret data transmission, copyright protection, and traceability. Despite promising results and numerous surveys proposed in the literature, there is still a lack of comprehensive analysis dedicated to deep learning-based image steganography and watermarking algorithms. In this paper, we focus on investigating three important aspects: neural networks, structure models, and training strategies. Our review covers the vast literature in this field. Furthermore, we provide a comprehensive statistical analysis from diverse perspectives, including models, loss functions, platforms, datasets, and attacks. Moreover, we conduct in a thorough comparative analysis and evaluation of existing representative algorithms, assessing their effectiveness within the context of deep learning. Finally, the challenges and potential research directions in the domain of deep-learning image steganography and watermarking algorithms are discussed to facilitate future research. • Comprehensive review of deep learning-based image steganography and watermarking. • In-depth analysis of neural networks, structure models, and training strategies. • Extensive statistical analysis of current models, datasets, attack, and platform. [ABSTRACT FROM AUTHOR]

Shoeprint reconstruction is essential in forensic science, but it is also challenging due to various inconsistencies in the patterns, textures, sizes, abrasions, etc. of shoeprints. The computational reconstruction of sharper and more complete shoeprints is conventionally conducted using handcrafted features, and it often requires human intervention. Prior studies using end-to-end machine learning approaches are limited in number and have not achieved a high level of performance. In this paper, we propose a model named ShoeRec, which employs variational autoencoder (VAE) as a component in a U-Net-like architecture to reconstruct missing regions and borders in shoeprint images. ShoeRec incorporates skip connections to preserve key patterns and employs VAE in the bottleneck to facilitate the reconstruction of desired shoeprints with the restoration of detail as perceived by humans. As a U-Net, the model skips the contextual information from the encoder to the decoder, and the compressed features in the latent space via VAE optimize the probabilistic distribution for reconstructing complete shoeprints. The reconstruction operation is automatically tuned according to the objective function, so as to reduce the structural correlation between the original and projected shoeprint and restore the absent information in the desired shoeprints. To the best of our knowledge, ShoeRec is the first deep learning infusion model that specializes in shoeprint reconstruction. The shoeprints reconstructed by ShoeRec have a close match with the originals in terms of both structures and patterns, and ShoeRec outperforms state-of-the-art generative models in human evaluation. • An end-to-end deep learning model focuses on shoeprint reconstruction for forensics. • Reconstruction of existing patterns together with the generation of missing regions. • Missing regions reconstruction is specifically designed based on the loss function. • The proposed model performs well for noisy, blurry, and partial input images. • Model outperforms state-of-the-art generative models in terms of human evaluation. [ABSTRACT FROM AUTHOR]

The data distribution shifting under varied working conditions impedes the application of deep learning models for fault diagnosis. Transfer learning can reduce the discrepancy in data distribution and improve model generalization from one condition with abundant labeled source data to another condition with unlabeled target data. However, existing transfer learning methods may hardly be applied to scenarios with few source domain data, and the insufficient source data hinder the establishment of the initial model. To address this issue, a deep semi-supervised transfer learning method with sensitivity-aware decision boundary adaptation (Sa-DBA) is proposed in this paper. The proposed triangular consistency in semi-supervised learning module can ensure the construction of the initial decision boundary. In transfer process, the Sa-DBA directly reduces the discrepancy of predicted matrixes between two domains to facilitate the adaptation of decision boundaries to the target data. Given that target data adjacent to decision boundaries may exhibit varying boundary sensitivities across different tasks, the nuclear-norm with task-specific parameter is introduced to further refine decision boundaries. Finally, extensive experiments on three diverse datasets, including CWRU, PHM, our own DPS test bench, and the best average accuracy reached 99.70%, 86.41%, and 88.59% respectively, which performed better than the other methods. • A deep semi-supervised transfer learning framework is proposed. • The triangular regularization is proposed in semi-supervised learning module. • Our method aligns predicted matrixes to reduce the domain discrepancy. • The task-weighted nuclear-norm is adopted to further refine decision boundaries. • Our method outperforms current methods on fault diagnosis with few source data. [ABSTRACT FROM AUTHOR]

This study provides a hybrid deep learning approach that enables the researchers to estimate the Tunnel Boring Machine (TBM) health status during excavation process using energy consumption data. By analyzing the energy forms and paths in the tunneling process, the key energy factors' consumption performance behavior affecting TBM safety is identified. The Ensemble Empirical Mode Decomposition (EEMD) method and a Convolution neural network-Long short term memory-Depth Neural Network (CNN-LSTM-DNN, CLDNN) model are integrated for Health Performance Parameter (HPP) prediction. A classification method is constructed by the Generative Adversarial Imputation Networks (GAIN) model for health status estimation. The SHapley Additive exPlanations (SHAP) method is performed to quantify the relationships between energy consumption and system health status. A tunneling case in Singapore is used to test the effectiveness of the developed methodology. It is found that (1) the proposed method provides TBM cutter wear and cooling oil prediction with a R-square (R2) of 0.8591 and 0.8727, respectively, and (2) classification results indicate that the estimation F1-score of 0.9863 can be achieved based on health status evaluation indicator (HSEI). The novelty of the proposed approach lies in its capabilities of (1) fully extracting and utilizing the energy consumption information in every regard to predict TBM health performance; (2) providing a good one-step-ahead health status estimation for the TBM excavation process. Therefore, this study proposes a feasible direction for the application of energy consumption data in TBM health status estimation. [ABSTRACT FROM AUTHOR]

Despite the great impact of lies in human societies and a meager 54% human accuracy for Deception Detection (DD), Machine Learning systems that perform automated DD are still not viable for proper application in real-life settings due to data scarcity. Few publicly available DD datasets exist and the creation of new datasets is hindered by the conceptual distinction between low-stakes and high-stakes lies. Theoretically, the two kinds of lies are so distinct that a dataset of one kind could not be used for applications for the other kind. Even though it is easier to acquire data on low-stakes deception since it can be simulated (faked) in controlled settings, these lies do not hold the same significance or depth as genuine high-stakes lies, which are much harder to obtain and hold the practical interest of automated DD systems. To investigate whether this distinction holds true from a practical perspective, we design several experiments comparing a high-stakes DD dataset and a low-stakes DD dataset evaluating their results on a Deep Learning classifier working exclusively from video data. In our experiments, a network trained in low-stakes lies had better accuracy classifying high-stakes deception than low-stakes, although using low-stakes lies as an augmentation strategy for the high-stakes dataset decreased its accuracy. • There is a clear distinction between low-stakes and high-stakes deception datasets. • Low-stakes lies may be used to classify high-stakes lies. • Combining low-stakes and high-stakes deception data does not yield good results. • Dataset bias is still an issue in video-based deception detection. • Use of automated deception detection in real-life settings is not recommended. [ABSTRACT FROM AUTHOR]

• This study focuses on unstructured road segmentation using advanced neural networks. • A dataset of mining roads based on different rock types is established. • Four semantic segmentation networks: FCN, UNet, PSPNet, and DeepLab v3+ are applied. • Study advances intelligent mining, focusing on road construction for unmanned driving. The mining industry is rapidly advancing towards automation and intelligence, with smart mines emerging as a future trend. Open-pit mining areas are semi-enclosed, and roads are essential for unmanned trucks to perceive the mining environment and execute various production tasks. The dynamic nature of open-pit mines, driven by production progress, leads to frequent alterations in roadways. As a consequence, roads become unstructured, with indistinct edges that easily blend into the surrounding mine environment. This poses a challenging operational environment for unmanned vehicles. To address this challenge in the realm of intelligent mining, this study establishes a dataset of mining roads based on different rock types and proposes an unstructured road segmentation method for mines by integrating residual networks, Contrast Limited Adaptive Histogram Equalization (CLAHE), and the Efficient Channel Attention (ECA) mechanism. This method is applied to four semantic segmentation networks: FCN, UNet, PSPNet, and DeepLab v3 +. The dataset and network model undergo validation using a specific hybrid loss function and relevant evaluation metrics. The results show that the established road dataset has good applicability, with an ablation experiment confirming the effectiveness of the added modules. This study introduces a new perspective for advancing unmanned driving in smart mines. [ABSTRACT FROM AUTHOR]

Boundary extraction is meaningful in medical image analysis since it explicitly extracts the tissue/lesions boundary coordinates, which benefits the follow-up diagnosis processes. However, accurately extracting human tissue/lesions is challenging due to unclear target boundaries or pseudo boundaries, which could lead to inferior performance of existing boundary extraction and/or semantic segmentation methods. We propose a Progressive Deep Snake (PDS), which is designed with a multi-step contour evolution (MCE) and a sequential label generation module (SLGM) to explicitly extract the smooth and continuous boundary of different tissue/lesions even if they are unclear, with irregular shape, or occluded by pseudo boundaries. (1) MCE is designed to initialize and evolve the contours to the target boundaries, where a multi-step deep supervision strategy is designed to improve flexibility and accuracy. (2) SLGM is designed to generate easy-to-difficult supervision signals for the contour evolution, which provides targets of moderate difficulty in each step for better guiding the contour evolution. Experiments on a challenging verbal segment datasets show that our method achieves state-of-the-art performance (MR_AVBCE dataset: mIoU: 0.9234, mDice: 0.9598, mBoundF: 0.9241, mWcov: 0.9704). Our code is available at https://github.com/sysu19351118/PDS-pytorch. [Display omitted] • We propose Progressive Deep Snake for extracting unclear medical image boundaries. • We propose MCE to enhance contour evolution with multi-step strategy. • We propose SLGM to generate easy-to-difficult signals for progressive evolution. [ABSTRACT FROM AUTHOR]

To ensure the timely detection of safety hazards in transmission lines and to enhance accurately the detection of insulator defects in complicated environments, this study proposes a 3D attention-focused pure convolutional target detection algorithm (CMYOLOv7) based on you only look once v7 (YOLOv7) for defect detection on insulators in complicated environments. Firstly, to address the incapacity of focusing on the target in the backbone feature extraction networks, this study proposes a focused pure convolutional feature extraction module (ConvSimCB) to enhance the extraction and focus capability on insulator defect features. Secondly, to solve the problem of loss of key detail feature information caused by maximum pooling in spatial pyramid pooling module (SPPCSPC), this study proposes a mixed spatial pyramid pooling module (MIXPCSPC) to retain abundant image texture detail information and increase accuracy in detection of tiny insulator defects. Finally, a lightweight generic upsampling operator (CARAFE) is introduced to enhance the feature map resolution to address image distortion caused by the Nearest Neighbor Method of upsampling. This study proposed CMYOLOv7 achieves 98.37% precision, 90.59% recall, 95.68% mean average precision (MAP), and 94% F1 score, higher than YOLOv7 by 0.61%, 8.64%, 5.41%, and 5%. [ABSTRACT FROM AUTHOR]

Medical image multi-object segmentation aims to accurately extract each object that is great significant for the medical image analysis. Although several methods based on deep learning have been widely developed, it is still challenging due to the similarities of different objects, the significant morphological variability, data heterogeneity, and the unbalanced distribution of samples. To address the above issues, we propose a novel Morphological Embedding Causal Constraint segmentation network (MCCSeg) for medical image multi-object segmentation. We build a hybrid CNN-Transformer encoder module to obtain local and global features. Specifically, a Causal Constraint Module (CCM) is proposed to decorrelate features by sample reweighting, which utilizes weight allocation to train the weight of training samples. The Random Fourier Features (RFF) is employed to solve nonlinear dependency problems and remove correlations between features to mitigate the influence of spurious correlations. The morphological guidance module (MG) is designed to extract the boundary as prior morphological information for enhancing feature representation, which is concatenated into the deep supervision module in the decoder for further optimizing feature extraction. The experiments demonstrate that MCCSeg outperforms other state-of-the-art methods, achieving improvements of 3.76% and 5.41% in terms of DICE and HD95 scores on Synapse dataset, respectively. The ablation experiments further verify the effectiveness of MCCSeg in improving generalization. [ABSTRACT FROM AUTHOR]

The open-source nature of Android, along with its coarse-grained permission management and widespread use, has heightened its vulnerability to malware threats. However, many traditional approaches face limitations in extracting comprehensive features, which hinders accurate and efficient Android malware detection. In this paper, we propose an effective hybrid approach combining an improved multi-scale convolutional neural network (MSCNN) with residual networks (ResNet) to defend against Android malware. The approach comprises an enhanced feature extraction network and a detection network. Initially, we introduce MSCNN, a novel deep learning model for comprehensive feature extraction. The MSCNN extends into three branches at distinct levels, concatenating features from low to high dimensions. This multilevel structure preserves rich semantic features while avoiding complex feature selection and analysis. Further, ResNet is employed as the detection network, and the hybrid models' performance is evaluated by comparison with a single ResNet. Finally, we validate the effectiveness of our approach by comparing our experimental results with state-of-the-arts. The experimental results show our approach effectively detects Android malware with high accuracy (99.20%) and precision (99.49%), and utilizing MSCNN as a multilevel feature extraction network significantly enhances the performance of the hybrid models, particularly, the F1-Score of MSCNN+ResNet18 increases by 4.8%. • Propose an improved MSCNN model extracting features from various data levels. • Design hybrid models combining MSCNN with GRU, ResNet18, ResNet34 and ResNet50. • MSCNN acts as upper feature extraction layer. • GRU, ResNet18, ResNet34 and ResNet50 as detection network, respectively. • Our approach detects Android malware, and enhances the hybrid model performance. [ABSTRACT FROM AUTHOR]

In U-turn bays near intersections, the conflict between U-turning vehicles and those going straight-ahead results in traffic accidents since straight-ahead vehicles cannot reliably anticipate the behavior of oncoming U-turning vehicles. However, previous studies on modeling U-turning behavior do not effectively capture the spatial–temporal interaction between the U-turning and surrounding vehicles. To address this issue, a deep-learning framework based on a temporal convolutional network (TCN) and multi-head attention mechanism is developed. The TCN is utilized to capture long-term dependencies of vehicles in the shared left- and U-turn lanes by extracting vehicle historical motion features. The self-attention mechanism extracts salient features related to the U-turn intentions, classifying the vehicles into left- and U-turning vehicles based on their driving intentions. A parallel TCN and spatial multi-head attention structure is constructed to model vehicle–vehicle interactions to further predict the future trajectory of U-turning vehicles. Finally, the obtained features are input into a Transformer-based decoder module and trajectory generator to predict the future displacement and body orientation of U-turning vehicles. The model is validated via comparison with state-of-the-art models and the observed trajectories under various scenarios. Ablation studies are conducted to quantify the efficacy of each module. Further, the effect of the surrounding homogenous and heterogeneous vehicles on U-turning vehicles in four different U-turn scenarios is quantified using spatial–temporal variation graphs and attention matrices. [ABSTRACT FROM AUTHOR]

Most existing deep learning-based polyp segmentation methods neglect two important aspects of polyps: the geometric orientation information of polyps and the point information of the entire colonoscopy area. In this paper, we introduce a multi-view orientational attention network (MVOA-Net), which incorporates orientation and point awareness to effectively address the issue of intra-class inconsistency resulting from variations in polyp shape, size, and position, as well as the inter-class indistinction caused by the high similarity between polyp lesions and surrounding tissues. To achieve robust orientation awareness, we propose a novel geometric orientation transformer encoder (GOTE) based on horizontal and vertical views. Moreover, To simultaneously capture the global context information of GOTE and emphasize the important local information of the convolution-based attention encoder (CBAE), a global and local cross attention fusion module (CAFM) is also proposed to simultaneously model the long-range dependencies of polyps and pay sufficient attention to the local boundaries of polyps. Additionally, a efficient atrous spatial pyramid pooling (E-ASPP) module is proposed to enhance the semantic representation of high-level features. Finally, a point-based affinity module (PBAM) and a multi-scale fusion module (MSFM) are proposed to distinguish the disguise of polyps, further alleviating inter-class indistinction. The ablation study results demonstrate the effectiveness of each component. Quantitative and qualitative experimental results show that MVOA-Net achieves the best segmentation accuracy across domain polyp datasets and has obvious advantages in segmenting multiple polyp objects. [ABSTRACT FROM AUTHOR]

In the contemporary digital landscape, the rising concern for mental health has sparked a surge in the use of mental health apps (MHAs) as accessible tools for addressing psychological well-being. Maintaining a high level of user satisfaction (USAT) is important for MHAs in the highly competitive app market. Leveraging BERT (Bidirectional Encoder Representations from Transformers), a state-of-the-art deep learning (DL) model, we perform topic modeling and sentiment analysis on 17,717 user online reviews. Specifically, we employ the BERTopic model to identify the determinants of USAT with MHAs. Utilizing a BERT-base-multilingual-uncase-sentiment model, we perform sentiment analysis to distinguish determinants that elicit satisfaction from those causing dissatisfaction. Also, this study tests and compares six machine learning (ML) algorithms to predict the influence of determinants on USAT with MHAs. The Light Gradient Boosting Machine (LightGBM) emerges as the top performer, showcasing its efficacy in predicting USAT determinants. By using SHAP (Shapley Additive exPlanations), an explainable ML model with cross-validation, we visualize the results of the LightGBM. The SHAP values show that the five most influential determinants of USAT with MHAs include soothing audio experience, smoking cessation support, payment and subscription management, tracking progress and mindful meditation experience. This study facilitates a deeper understanding of user experiences through the identification and prediction of determinants of USAT with MHAs. Understanding these factors and their interplay is essential for developers, clinicians, and stakeholders who aim to enhance MHAs' services and ultimately improve the well-being of users. [ABSTRACT FROM AUTHOR]

This paper presents a novel deep learning model called Self-Attention Layer within a Convolutional Neural Network (SACNN), specifically designed for detecting acoustic data in extensive datasets containing emergency vehicle sirens and road noise. SACNN leverages a combination of EfficientNet and One Dimensional Convolutional Neural Networks (1D_CNN) to improve the precision and efficiency of detection. The dataset comprises 3-second WAV-format audio files categorized into Emergency Vehicles (Ambulance and Firetruck sirens) and Traffic (plain traffic sounds), with each category having 200 sound files and their corresponding 200 spectrogram images per audio file. Experimental results demonstrate SACNN's superior performance across various metrics, including accuracy, precision, recall, and F1 score, while also achieving notable advancements in computational efficiency. The research addresses the need for more effective solutions in the domain of emergency vehicle siren and road noise detection by introducing SACNN as a cutting-edge solution. By integrating EfficientNet and 1D_CNN, SACNN surpasses existing models in accuracy and computational efficiency. Comparative analysis with EfficientNet and 1D_CNN using a large-scale dataset validates SACNN's superior performance, with an average accuracy, precision, recall, and F1 score of 1.00. Additionally, SACNN exhibits the highest computational efficiency, averaging 0.20883 s per sample. These findings highlight SACNN's efficacy in detecting and classifying emergency vehicle sirens and road noise in large-scale datasets, with implications for traffic management, public safety, and noise pollution monitoring. SACNN offers the potential for constructing real-time acoustic data detection systems. [ABSTRACT FROM AUTHOR]

Due to their high performance in modeling and forecasting a large amount of real-world complex phenomena, deep convolutional neural networks have received a great deal of attention over the past ten years. It has been extensively utilized in several recent vision problems, such as image denoising and deblurring. By incorporating recent developments in learning-based nonlinear diffusion models and highlighting the connection between partial differential equations (PDEs) and networks, especially residual networks, we propose a flexible learning reaction–diffusion system with spatially computed convolution kernels. This trained PDE is employed to prevent the low-level features from deteriorating, which is needed for tasks involving denoising. In fact, we introduce an improved network architecture for a denoising network entitled VPDNet through an optimal control problem. More precisely, the network architecture is governed by an explicit Euler discretization of a reaction–diffusion system. We give the main algorithm to identify the kernel parameters using the Alternating Direction Method of Multipliers (ADMM) combined with a Primal–Dual algorithm. Finally, numerical simulations are provided with appropriate comparisons to demonstrate the efficiency of the elaborated network. • We treat a variational PDNet for image denoising problem. • We propose a propose flexible learning reaction–diffusion system. • The introduced network and the proposed procedure achieves superior performance comparing with other denoising techniques. [ABSTRACT FROM AUTHOR]