Search

Road Condition Monitoring is a critical task for the management and maintenance of the pavement network infrastructure by the authorities. In recent years, the application of Artificial Intelligence (AI) techniques in this domain has experienced a significant growth, driven by the continuous advancements in AI algorithms. This paper presents a comprehensive review of the latest developments in Road Condition Monitoring approaches using AI methods, with a particular focus on Deep Learning techniques, covering works published from 2020 onwards. It highlights novel approaches that have not been thoroughly explored in previous literature reviews. The literature review categorizes studies based on the type of signal data, distinguishing between acoustic, vibrational, and vision-based approaches. For each data type, the paper examines and discuss the most recent advancements and improvements achieved through AI techniques. Additionally, it provides an overview of future directions and identifying key challenges that remain open in the field. In conclusion, relatively few studies have focused on the analysis of acoustic data, although some studies have reported promising results. Methods based on vibrational data typically integrate feature extraction in frequency and wavelet domain with Convolutional Neural Networks or Long Short-Term Memory Networks. Meanwhile, vision-based methods have experienced significant improvements, driven by the constant evolution of Deep Learning architectures. A total of 173 research articles are summarized across 10 tables.

Class activation maps (CAMs) are powerful tools for better understanding what convolutional neural networks learn and the reliability of their learning capability within relevant contexts. Highly inspired by Grad-CAM and Score-CAM, this manuscript proposes a novel method called Increment-CAM, which overcomes the limitations of both methods and enhances the localization capability of heatmaps, suppressing false activations to provide better discriminative visualizations. This paper offers a three-phase approach, generating activations through Grad-CAM in Phase 1 and applying a Score-CAM-like strategy to the Grad-CAM activation in Phase 2 with the difference that the input image is downsampled to the size of Grad-CAM activations to boost the computational process significantly. Finally, in Phase 3, the activations obtained in Phase 1 are combined with those generated in Phase 2, followed by regularizing the weighted sum. The steps involved in the Increment-CAM approach include Grad-CAM computation, Score-CAM computation with a modified approach, heatmap merging, and regularizing to obtain the final activation map. The proposed approach offers improved localization capabilities, better visualizations, and reduced false activations, providing fast, more accurate and interpretable visualizations of critical regions in an image. The paper also describes the experimental setup, evaluation metrics, and results of the proposed method on ImageNet, PascalVOC2007 and Caltech-UCSD Birds-200-2011 datasets, demonstrating its effectiveness compared to the existing visualization techniques.

Diabetic Retinopathy is one of the leading global causes of vision impairment and blindness in humans. It has seen a rise in prevalence, necessitating the development of advanced automatic detection methods. This paper presents a survey of the evolution in deep learning techniques for diabetic retinopathy detection, emphasizing the transition from traditional machine learning to sophisticated deep learning architectures such as convolutional neural networks. It discusses the role of transfer learning, end-to-end learning, and hybrid models in overcoming medical detection challenges while highlighting the need for artificial intelligence interpretability and real-time screening integration in clinical workflows. Building on this survey, the paper introduces a focused study on cross-dataset deployment of transfer learning for diabetic retinopathy detection and grading. Consequently, this paper evaluates 26 pre-trained models from various convolutional neural network families to provide a comprehensive comparison between the state-of-the-art CNN architectures in the field. Additionally, this study also employs Grad-CAM visualization to interpret the model’s decision-making, bridging advanced artificial intelligence techniques with practical healthcare applications for diabetic retinopathy.

Industrial robots are prone to failure due to harsh working environments, which affects movement accuracy. The fault diagnosis of industrial robots has become an indispensable part of robot collaborative maintenance in intelligent manufacturing. Most existing diagnostic methods only use a single data source, and the diagnostic accuracy will be affected due to signal acquisition errors and noise interference. This paper proposes a multi-source data fusion and channel attention convolutional neural network (MD-CA-CNN) for fault diagnosis of multi-joint industrial robots. The network takes the time domain data and time-frequency domain data of the vibration signal, torque signal, and current signal of the six joints of the robot as input. Then, we realize the diagnosis of the faults by using a Softmax Classifier layer after the two parts of feature extraction and feature fusion. In addition, a channel attention mechanism is developed. It acts on the two parts of feature extraction and feature fusion, respectively. It assigns weights to different source data and weights to time-domain and time-frequency domain features. Finally, we established a test bench to compare the proposed method with the deep learning algorithm that only uses multi-data source fusion, the deep learning algorithm that only uses a single data source, and the commonly used machine learning algorithm. The results show that the MD-CA-CNN model proposed in this paper has the highest accuracy and stability, reaching 95.8% $\pm ~0.39$ %, which verifies the method’s effectiveness.

Skin cancer recognition poses a significant challenge in the field of deep learning. While conventional convolutional neural networks have been extensively employed for classifying skin cancer images, their fixed receptive field limits their ability to capture the global features present in such images. Conversely, transformer-based models that rely on self-attention can effectively model long-range dependencies, but they come with high computational complexity and exhibit certain limitations in local feature induction. To address this issue, this paper presents a novel skin cancer recognition network named ECRNet. ECRNet has been designed to effectively capture both global and local information, and it introduces an explicit vision center to accomplish this purpose. Moreover, this paper presents a feature fusion module known as the CCPA block. This module utilizes both coordinate attention and channel attention mechanisms to extract image features and enhance the representation of skin cancer images. To evaluate the performance of ECRNet, extensive experimental comparisons were conducted on the ISIC2018 dataset. The experimental results demonstrate that ECRNet outperforms the baseline model, showing improvements of 1.19% in accuracy (ACC), 1.96% in precision, 4.08% in recall, and 3.28% in the F1 score.

In order to monitor the growth and development of tomatoes, and improve the efficiency of flower and fruit thinning and tomato picking, this paper constructs a tomato flower and fruit dataset and proposes a TF-YOLOv5s model for the detection of tomato flowers and fruits in natural environments. Based on the YOLOv5s model, a C3Faster module is introduced to reduce the number of parameters and calculations while maintaining detection accuracy. The regular convolution is replaced by depth-wise separable convolution (DWConv) to avoid parameter redundancy. To improve the convergence and accuracy of the model, this paper replaces Complete Intersection over Union (CIoU) loss with Efficient Intersection over Union (EIoU) loss. The Squeeze-and-Excitation (SE) module is added to improve the model’s sensitivity to the features of the tomato flowers and fruits. Compared with the baseline model, the number of parameters is reduced by 54.5%, the weight file is reduced by 52.8%, the Floating-point Operation Per second (FLOPs) is reduced by 48.7%. The detection accuracy of tomato flowers and fruits mAP@0.5 has improved by 1.4% and 1.2% respectively. TF-YOLOv5s is used to detect three types of targets: tomato flowers, red tomatoes, and green tomatoes, and mAP@0.5 of which can reach as high as 95.2%. Furthermore, the improved algorithm is deployed on two edge computing devices to verify its effectiveness. Experimental results show that the algorithm in this paper can achieve high detection with less computational resources. This algorithm has the potential value of application in practical tomato production.

Most power systems’ approaches are currently tending towards stochastic and probabilistic methods due to the high variability of renewable sources and the stochastic nature of loads. Conventional power flow (PF) approaches such as forward-backward sweep (FBS) and Newton-Raphson require a high number of iterations to solve non-linear PF equations making them computationally very intensive. PF is the most important study performed by utility, required in all stages of the power system, especially in operations and planning. This paper discusses the applications of deep learning (DL) to predict PF solutions for three-phase unbalanced power distribution grids. Three deep neural networks (DNNs); Radial Basis Function Network (RBFnet), Multi-Layer Perceptron (MLP), and Convolutional Neural Network (CNN), are proposed in this paper to predict PF solutions. The strength of the proposed DNN models over the traditional iterative-based PF solvers is that these models can capture the nonlinear relationships in PF calculations to accurately predict the solutions. The PF problem is formulated as a multi-output regression model where two or more output values are predicted based on the inputs. The training and testing data are generated through the OpenDSS-MATLAB COM interface. These methods are completely data-driven where the training relies on reducing the mismatch at each node without the need for the knowledge of the system. The novelty of the proposed methodology is that the models can accurately predict the PF solutions for the unbalanced distribution grids with mutual coupling and are robust to different R/X ratios, topology changes as well as generation and load variability introduced by the integration of distributed energy resources (DERs) and electric vehicles (EVs). To test the efficacy of the DNN models, they are applied to IEEE 4-node and 123-node test cases, and the American Electric Power (AEP) feeder model. The PF results for RBFnet, MLP, and CNN models are discussed in this paper which demonstrate that all three DNN models provide highly accurate results in predicting PF solutions.

The application of Deep Learning models in fruit analysis has garnered significant attention due to its potential to revolutionize the agricultural sector and enhance crop monitoring. This paper presents a comprehensive review of recent research efforts in fruit analysis using Deep Learning techniques. The study delves into model selection, dataset characteristics, evaluation metrics, challenges, and future directions in this domain. Various model architectures, including classical Convolutional Neural Networks (CNNs) and advanced detection models like R-CNN and YOLO, have been explored for tasks ranging from fruit classification to detection. Evaluation metrics such as precision, recall, F1-score, and mean Average Precision (mAP) have been commonly used to assess model performance. Challenges, including data scarcity, labeling complexities, variations in fruit characteristics, and computational efficiency, have been identified and discussed. The paper also presents an overview of available datasets, encompassing both proprietary and publicly accessible sources. Future research directions involve exploring enhanced data augmentation, multi-modal integration, knowledge transfer across species, robustness in dynamic environments, improved computational efficiency, and practical integration of models into real-world agricultural systems. This review provides valuable insights for researchers and practitioners aiming to leverage Deep Learning for fruit analysis in the pursuit of sustainable agriculture and food production.

In this paper, we propose a data augmentation technique based on Convolutional Neural Networks (CNN or ConvNet) training to efficiently obtain a dataset of images containing concrete cracks. Concrete cracks usually do not have a standardized shape and have complex patterns, making it difficult to obtain images of them, and there is a risk of exposure to dangerous situations when securing data. Therefore, in this paper, we efficiently address the difficulty of dataset collection by using a data augmentation technique based on learning the direction and thickness of cracks, which is cost-effective and time-efficient. Moreover, to improve efficiency, we introduce a method of adaptively handling crack data by constructing a quadtree based on the presence of cracks. To confirm the extent of the improvement in accuracy, we conducted experiments applying the crack detection algorithm to various scenes, and the accuracy was improved in all scenes when measured by IoU (Intersection over union) accuracy. When the algorithm was performed without augmenting the crack data, the false detection rate was about 25%. However, when we augmented the data using our method, the false detection rate significantly decreased to 3%.

This paper presents an end-to-end trainable wavelet video coder based on motion-compensated temporal filtering. Thereby, it introduces a different coding scheme for learned video compression, which is dominated by residual and conditional coding approaches. By performing discrete wavelet transforms in temporal, horizontal, and vertical dimensions, an explainable framework with spatial and temporal scalability is obtained. This paper investigates a novel trainable motion-compensated temporal filtering module implemented using the lifting scheme. It demonstrates how multiple temporal decomposition levels can be considered during training. Furthermore, larger temporal displacements owing to the coding order are addressed and an extension adapting to different motion strengths during inference is introduced. The experimental analysis compares the proposed approach to learning-based coders and traditional hybrid video coding. Especially at high rates, the approach exhibits promising rate-distortion performance. The proposed method achieves average Bjøntegaard Delta savings of up to 21% over HEVC, and outperforms state-of-the-art learned video coders.

Manually building neural network models is a great test of researchers’ knowledge reserves, so using Neural Architecture Search (NAS) to automatically construct neural networks is becoming increasingly popular. This paper uses an improved Differentiable Architecture Search (DARTS) to automatically build a classification network model, DARTS is a gradient-based NAS algorithm. However, DARTS uses local selection, so this paper proposes a selection algorithm based on global selection. The improved algorithm can ensure that all operations connected to the same intermediate node can be fairly compared, ensuring fairness in selection and searching for more types of neural network architectures. DARTS only uses fixed candidate operations to construct the neural network, so the structure of the neural network is relatively single, skip connection will dominate the searched network, and DARTS uses a large number of approximation strategies, which can easily lead to a decrease in the accuracy of the model. For the problems, we add a SENet attention mechanism after cell output, SENet can extract features on the feature layer in the channel dimension, it can not only improve the search performance of the network but also effectively increase the diversity and robustness of the network. The final test error on CIFAR10 reaches 2.48%.

Brain age prediction is important for understanding brain development and aging. Currently, researchers can predict brain age using resting-state functional MRI (rs-fMRI) data. However, there are differences in brain age development among different subjects, and the same subject also has different development at different ages. So far, how to accurately estimate brain age using rs-fMRI efficiently remains a challenging problem. Therefore, a brain age prediction model with the similarity metric convolutional neural network is proposed in this paper. Specifically, this paper first introduces a siamese convolutional neural network, which includes convolution, batch normalization, and pooling steps simultaneously learns the features of two groups of rs-fMRI, and designs a similarity measurement network. Subsequently, fMRI images of two groups of different subjects are input into the network, and a similarity metirc module is designed to calculate the similarity between the two groups of images. Then the network is optimized by a loss function, and finally, the average value of the three groups of sample labels with the greatest similarity is taken. The absolute mean error and correlation coefficients obtained from the model are 5.337 and 0.6279, respectively. Experimental results show that this method has low mean absolute error and high correlation coefficient on the longitudinal imaging data set of Southwest University.

Brain-inspired architectures are gaining increased attention, especially for edge devices to perform cognitive tasks utilizing its limited energy budget and computing resources. Hyperdimensional computing (HDC) paradigm is an emerging framework inspired by an abstract representation of neuronal circuits’ attributes in the human brain. That includes a fully holographic random representation, high-dimension vectors representing data, and robustness to uncertainty. The basic HDC pipeline consists of an encoding, training and comparison stages. The encoding algorithm maps different representations of inputs into a single class and stores them in the associative memory (AM) throughout the training stage. Later, during the inference stage, the similarity is computed between the query vector, which is encoded using the same encoding model, and the stored classes in the AM. HDC has shown promising results for 1D applications using less power, and lower latency than state-of-the-art digital neural networks (DNN). While in 2D applications, convolutional neural network (CNN) still achieves higher classification accuracy at the expense of more computations. In this paper, a comprehensive study on the HDC paradigm, main algorithms, and its implementation is presented. Moreover, the main state-of-the-art HDC architectures for 1D and 2D applications are highlighted. The article also analyzes two competing paradigms, namely, HDC and CNN, in terms of accuracy, complexity, and the number of operations. The paper concluded by highlighting challenges and recommendations for future directions on the HDC framework.

Design of a vision-based traffic analytic system for urban traffic video scenes has a great potential in context of Intelligent Transportation System (ITS). It offers useful traffic-related insights at much lower costs compared to their conventional sensor based counterparts. However, it remains a challenging problem till today due to the complexity factors such as camera hardware constraints, camera movement, object occlusion, object speed, object resolution, traffic flow density, and lighting conditions etc. ITS has many applications including and not just limited to queue estimation, speed detection and different anomalies detection etc. All of these applications are primarily dependent on sensing vehicle presence to form some basis for analysis. Moving cast shadows of vehicles is one of the major problems that affects the vehicle detection as it can cause detection and tracking inaccuracies. Therefore, it is exceedingly important to distinguish dynamic objects from their moving cast shadows for accurate vehicle detection and recognition. This paper provides an in-depth comparative analysis of different traffic paradigm-focused conventional and state-of-the-art shadow detection and removal algorithms. Till date, there has been only one survey which highlights the shadow removal methodologies particularly for traffic paradigm. In this paper, a total of 70 research papers containing results of urban traffic scenes have been shortlisted from the last three decades to give a comprehensive overview of the work done in this area. The study reveals that the preferable way to make a comparative evaluation is to use the existing Highway I, II, and III datasets which are frequently used for qualitative or quantitative analysis of shadow detection or removal algorithms. Furthermore, the paper not only provides cues to solve moving cast shadow problems, but also suggests that even after the advent of Convolutional Neural Networks (CNN)-based vehicle detection methods, the problems caused by moving cast shadows persists. Therefore, this paper proposes a hybrid approach which uses a combination of conventional and state-of-the-art techniques as a pre-processing step for shadow detection and removal before using CNN for vehicles detection. The results indicate a significant improvement in vehicle detection accuracies after using the proposed approach.

Manual inspection of textiles is a long, tedious, and costly method. Technology has solved this problem by developing automatic systems for textile inspection. However, Jacquard fabrics present a challenge because patterns can be complex and seemingly random to systems. Only a few in-depth studies have been conducted on jacquard fabrics despite their important and intriguing nature. Previous studies on jacquard fabrics are of simple patterns. This paper introduces a new and novel field in fabrics defect detection. Complex-patterned jacquard fabrics are much more challenging. In this paper, novel defect detection models for jacquard-patterned fabrics are presented. Owing to the lack of available databases for jacquard fabrics, we compiled and experimented on our own novel dataset. Our dataset was collected from plain, undyed jacquard fabrics with different complex patterns. In this study, we used and tested several deep learning models with image pre-processing and convolutional neural networks (CNNs) for unsupervised detection of defects. We also used multispectral imaging, combining normal (RGB) and near-infrared (NIR) imaging to improve our system and increase its accuracy. We propose two systems: a semi-manual system using a simple CNN network for operation on separate patterns and an integrated automated system that uses state-of-the-art CNN architectures to run on the entire dataset without prior pattern specification. The images are preprocessed using contrast-limited adaptive Histogram Equalization (CLAHE) to enhance their features. We concluded that deep learning is efficient and can be used for defect detection in complex patterns. Proposed method of EfficientNet CNN gave high accuracy reaching 99% approximately. We also found that multispectral imaging is more advantageous and yields higher accuracy.

The development of intelligent technologies gains popularity in the education field. The rapid growth of educational data indicates traditional processing methods may have limitations and distortion. Therefore, reconstructing the research technology of data mining in the education field has become increasingly prominent. In order to avoid unreasonable evaluation results and monitor the students’ future performance in advance, this paper comprehensively uses the relevant theories of clustering, discrimination and convolution neural network to analyze and predict students’ academic performance. Firstly, this paper proposes that the clustering-number determination is optimized by using a statistic which has never been used in the algorithm of K-means. Then, the clustering effect of K-means algorithm is tested by discriminant analysis. The convolutional neural network is introduced for training and testing data that are labeled with categories. The generated model can be used to predict prospective performance. Finally, in order to validate the prediction results, the effectiveness of the generated model is evaluated by using two metrics in two cross-validation methods. The experimental result demonstrates that the statistic not only solves the difficulty to determine the clustering number in K-means algorithm from an objective and quantitative point of view, but also improves the reliability of prediction results.

Due to a growing number of people who carry out various adrenaline activities or adventure tourism and stay in the mountains and other inaccessible places, there is an increasing need to organize a search and rescue operation (SAR) to provide assistance and health care to the injured. The goal of SAR operation is to search the largest area of the territory in the shortest time possible and find a lost or injured person. Today, drones (UAVs or drones) are increasingly involved in search operations, as they can capture a large, controlled area in a short amount of time. However, a detailed examination of a large amount of recorded material remains a problem. Even for an expert, it is not easy to find searched people who are relatively small considering the area where they are, often sheltered by vegetation or merged with the ground and in unusual positions due to falls, injuries, or exhaustion. Therefore, the automatic detection of persons and objects in images/videos taken by drones in these operations is very significant. In this paper, the reliability of existing state-of-the-art detectors such as Faster R-CNN, YOLOv4, RetinaNet, and Cascade R-CNN on a VisDrone benchmark and custom-made dataset SARD build to simulate rescue scenes was investigated. After training the models on selected datasets, detection results were compared. Because of the high speed and accuracy and the small number of false detections, the YOLOv4 detector was chosen for further examination. YOLOv4 model results related to different network sizes, different detection accuracies, and transfer learning settings were analyzed. The model robustness to weather conditions and motion blur were also investigated. The paper proposes a model that can be used in SAR operations because of the excellent results in detecting people in search and rescue scenarios.

Despite the growing interest in the interplay of machine learning and optimization, existing contributions remain scattered across the research board, and a comprehensive overview on such reciprocity still lacks at this stage. In this context, this paper visits one particular direction of interplay between learning-driven solutions and optimization, and further explicates the subject matter with a clear background and summarized theory. For instance, machine learning and its offsprings are trending because of their enhanced capabilities in automating analytical modeling. In this realm, learning-based techniques (supervised, unsupervised, and reinforcement) have grown to complement many of the optimization problems in testing and training. This paper overviews how machine learning-based techniques, namely deep neural networks, echo-state networks, reinforcement learning, and federated learning, can be used to solve complex and analytically intractable optimization problems, for which specific cases are examined in this paper. The paper particularly overviews when learning-based algorithms are useful at solving particular optimizing problems, especially those of random, dynamic, and mathematically complex nature. The paper then illustrates such applications by presenting particular use-cases in communications and signal processing including wireless scheduling, wireless offloading and resource management, power control, aerial base station placement, virtual reality, and vehicular networks. Lastly, the paper sheds light on some future research directions, where the dynamicity and randomness of the underlying optimization problems make deep learning-driven techniques a necessity, namely in sensing at the terahertz (THz) bands, cellular vehicle-to-everything, 6G communication networks, underwater optical networks, distributed optimization, and applications of emerging learning-based techniques.

The work in this paper proposes the application of the pinball quantile loss function to guide a deep neural network for Non-Intrusive Load Monitoring. The proposed architecture leverages concepts such as Convolution Neural Networks and Recurrent Neural Networks. For evaluation purposes, this paper also presents a set of complementary performance metrics for energy estimation. Finally, this paper also reports on the results of a comprehensive benchmark between the proposed network and three alternative deep neural networks, when guided by the pinball and Mean Squared Error loss functions. The obtained results confirm the disaggregation superiority of the proposed system, while also showing that the performances obtained using the pinball loss function are consistently superior to the ones obtained using the Mean Squared Error loss.

This paper reviews studies on neural networks in aerodynamic data modeling. In this paper, we analyze the shortcomings of computational fluid dynamics (CFD) and traditional reduced-order models (ROMs). Subsequently, the history and fundamental methodologies of neural networks are introduced. Furthermore, we classify the neural networks based studies in aerodynamic data modeling and illustrate comparisons among them. These studies demonstrate that neural networks are effective approaches to aerodynamic data modeling. Finally, we identify three important trends for future studies in aerodynamic data modeling: a) the transformation method and physics informed models will be combined to solve high-dimensional partial differential equations; b) in the research area of steady aerodynamic response predictions, model-oriented studies and data-integration-oriented studies will become the future research directions, while in unsteady aerodynamic response predictions, radial basis function neural networks (RBFNNs) are the best tools for capturing the nonlinear characteristics of flow data, and convolutional neural networks (CNNs) are expected to replace long short-term memories (LSTMs) to capture the temporal characteristics of flow data; and c) in the field of steady or unsteady flow field reconstructions, the CNN-based conditional generative adversarial networks (cGANs) will be the best frameworks in which to discover the spatiotemporal distribution of flow field data.

Monitoring single-channel EEG is a promising home-based approach for insomnia identification. Currently, many automatic sleep stage scoring approaches based on single-channel EEG have been developed, whereas few studies research on automatic insomnia identification based on single-channel EEG labelled with sleep stage annotations. In this paper, we propose a one-dimensional convolutional neural network (1D-CNN) model for automatic insomnia identification based on single-channel EEG labelled with sleep stage annotations, and further investigate the identification performance based on different sleep stages EEG epochs. Single-channel EEG on 9 insomnia patients and 9 healthy subjects was used in this study. We constructed 4 subdatasets from EEG epochs based on the sleep stage annotations: All sleep stage dataset (ALL-DS), REM sleep stage dataset (REM-DS), light sleep stage dataset (LSS-DS), and SWS sleep stage dataset (SWS-DS). Subsequently, 4 subdatasets were fed into our 1D-CNN. We conducted experiments under intra-patient and inter-patient paradigms, respectively. Our experiments demonstrated that our 1D-CNN leveraging 3 subdatasets composed of REM, LSS and SWS epochs, respectively, achieved higher average accuracies in comparison with baseline methods under both intra-patient and inter-patient paradigms. The experimental results also indicated that amongst all the sleep stages, 1D-CNN leveraging REM and SWS epochs exhibited the best insomnia identification average accuracies in intra-patient paradigm, which are 98.98% and 99.16% respectively, whereas no statistically significant difference was found in inter-patient paradigm. For automatic insomnia identification based on single-channel EEG labelled with sleep stage annotations, 1D-CNN model introduced in this paper could achieve superior performance than traditional methods.

Person re-identification, identifying and tracking pedestrians in cross-domain monitoring systems, is an important technology in the computer vision field and of real significance for the construction of smart cities. With the development of deep learning techniques, especially convolutional neural networks, this technology has received more extensive attention and improvement in recent years and a large number of noteworthy achievements have emerged. This paper provides a comprehensive overview of person re-identification approaches to assist researchers in quickly understand this field with preference as well as to provide a more structured framework. By reviewing more than 300 re-identification related papers, the focus of these studies is summarized as information extraction, metric learning, post-processing, efficiency improvement, labeling cost reduction, and data type expansion. This classification is then organized based on different technologies, and on this basis, the pros and cons of each technology are analyzed. Moreover, this overview summarizes the difficulties and challenges of re-identification and discusses the possible research directions for reference.

In this paper, we propose Instance Segmentation Detector (ISD) to extract the enhanced feature-maps under the situations where training dataset is limited in the specific industry domain such as semiconductor photo lithography inspection. ISD is used as a new backbone network of state-of-the-art Mask R-CNN framework for instance segmentation. ISD consists of four dense blocks and four transition layers. Each dense block in ISD has the shortcut connection and the concatenation of the feature-maps produced in layer with dynamic growth rate. ISD is trained from scratch without using recently approached transfer learning method. Additionally, ISD is trained with image dataset pre-processed by means of the specific designed image filter to extract the better enhanced feature map of Convolutional Neural Network (CNN). In ISD, one of the key principles is the compactness, plays a critical role for addressing real time problem and for application on resource bounded devices. To validate the model, this paper uses the real image collected from the computer vision system embedded in the currently operating semiconductor manufacturing equipment. ISD achieves consistently better results than state-of-the-art methods at the standard mean average precision. Specifically, our ISD outperforms baseline method DenseNet, while requiring only 1/4 parameters. We also observe that ISD can achieve comparable better results than ResNet, with only much smaller 1/268 parameters, using no extra data or pre-trained models.

Entity disambiguation refers to the accurate inference of the real mention of an entity with the same name according to the context. Most existing studies focused on long texts, for short texts, the performance has been unsatisfactory due to sparsity. In this paper, we treat the entity disambiguation task as a classification problem. we propose a novel neural network-based capsule network and convolutional neural network for entity disambiguation, leveraging full semantic information of short text data. In particular, a self-attention mechanism is utilized to further filter the semantic information extracted from the capsule network. On the other hand, a convolutional neural network with combined pooling is established to capture semantics from another channel. In the end, the semantic features obtained by the above models are combined through a fully connected layer to complete the task of entity disambiguation. The experimental results on the CCKS 2019 entity linking dataset showed that the dual-channel hybrid network proposed in this paper achieved an F1-score of 88.04%, which is superior to that of the existing mainstream deep learning model, thereby verifying the effectiveness of the model.

High-level feature maps of Convolutional Neural Networks are computed by reusing their corresponding low-level feature maps, which brings into full play feature reuse to improve the computational efficiency. This form of feature reuse is referred to as feature reuse between convolutional layers. The second type of feature reuse is referred to as feature reuse within the convolution, where the channels of the output feature maps of the convolution are computed by reusing the same channels of the input feature maps, which results in an approximation of the channels of the output feature maps. To compute them accurately, we need specialized input feature maps for every channel of the output feature maps. In this paper, we first discuss the approximation problem introduced by full feature reuse within the convolution and then propose a new feature reuse scheme called Reducing Approximation of channels by Reducing Feature reuse (REAF). The paper also shows that group convolution is a special case of our REAF scheme and we analyze the advantage of REAF compared to such group convolution. Moreover, we develop the REAF+ scheme and integrate it with group convolution-based models. Compared with baselines, experiments on image classification demonstrate the effectiveness of our REAF and REAF+ schemes. Under the given computational complexity budget, the Top-1 accuracy of REAF-ResNet50 and REAF+-MobileNetV2 on ImageNet will increase by 0.37% and 0.69% respectively. The code and pre-trained models will be publicly available.

The goal of video super-resolution technique is to address the problem of effectively restoring high-resolution (HR) videos from low-resolution (LR) ones. Previous methods commonly used optical flow to perform frame alignment and designed a framework from the perspective of space and time. However, inaccurate optical flow estimation may occur easily which leads to inferior restoration effects. In addition, how to effectively fuse the features of various video frames remains a challenging problem. In this paper, we propose a Local-Global Fusion Network (LGFN) to solve the above issues from a novel viewpoint. As an alternative to optical flow, deformable convolutions (DCs) with decreased multi-dilation convolution units (DMDCUs) are applied for efficient implicit alignment. Moreover, a structure with two branches, consisting of a Local Fusion Module (LFM) and a Global Fusion Module (GFM), is proposed to combine information from two different aspects. Specifically, LFM focuses on the relationship between adjacent frames and maintains the temporal consistency while GFM attempts to take advantage of all related features globally with a video shuffle strategy. Benefiting from our advanced network, experimental results on several datasets demonstrate that our LGFN can not only achieve comparative performance with state-of-the-art methods but also possess reliable ability on restoring a variety of video frames. The results on benchmark datasets of our LGFN are presented on https://github.com/BIOINSu/LGFN and the source code will be released as soon as the paper is accepted.

Large-scale image datasets with numerous occlusion patterns prevail in real applications. The classification scheme based on subspace decomposition-based estimation with squared l2 -norm regularization (SDBE_L2) has shown promising performance for the classification of partially occluded images. For the large-scale image datasets with numerous occlusion patterns, it however suffers from a high labor intensity in acquiring extra image pairs and a large consumption of computational resources in the training stage. To reduce the labor intensity, this paper enumerates several useful types of extra image pairs to guide the collection of extra images and introduces an intra-class random pairing method to semi-automatically form the extra image pairs. To alleviate the consumption of computational resources, this paper proposes two dictionary compression approaches: 1) uncentered PCA-based single partition compression (UPSPC), which compresses the dictionary to a size not larger than twice the column vector length without affecting the classification accuracy, and 2) uncentered PCA-based intra-class partition compression (UPIPC), which can further shrink the occlusion error dictionary (or class dictionary) when it has a small number of occlusion classes (or image classes). The proposed approaches are based on the property of SDBE_L2 being invariant to the uncentered PCA of sub-dictionaries. The extensive experiments on the Caltech-101 dataset and Oxford-102 flower dataset demonstrate the enumerated examples and the intra-class random pairing method facilitate acquiring the extra images and forming the extra image pairs only with a small loss in the classification accuracy. The experimental results on a large-scale occluded image dataset synthesized from the ILSVRC 2012 classification dataset with numerous occlusion patterns show that the proposed dictionary compression approaches reduce the dictionary size by over 11 times and shorten the training time by more than 39 times without loss in the classification accuracy.

Nowadays, crowd analysis is one of the most important concepts that needs be relied upon, it contributes to decision making and ensuring the safety and security of the crowd. There are a variety of interesting research problems within the scope of crowd analysis including crowd tracking, crowd behaviour recognition and crowd counting. Crowd counting based on images and videos has been studied in past years. Nonetheless, estimating and detecting the number of human heads remains a challenging task due to occlusions, resolution, and lighting changes. This paper provides an overview and performance comparison of crowd counting techniques using convolutional neural networks (CNN) based on density map estimation. In this paper, we present a comprehensive analysis and benchmarking of crowd counting based on the UCF-QNRF dataset that contains the largest number of crowd count images and head annotations available in the public domain. We also show the density maps generation and their empirical evaluation along with performance comparison.

Constructing a robust appearance model of the visual object is a crucial task for visual object tracking. Recently, more and more studies combine spatial feature with a temporal feature to improve the tracking performance. These methods successfully apply the features from spatial and temporal to address the problem for tracking. This paper presents a novel method for visual object tracking based on spatiotemporal feature combined with correlation filters. In this paper, the visual features of a target object are extracted from a spatial-temporal residual network (STResNet) appearance model with two sub-networks. The STResNet appearance model learns separately spatial feature and temporal feature, respectively, so that we can effectively utilize spatial context around the surrounding of the target object in each frame and the temporal relationship between successive frames to refine the appearance representation of the target object. Finally, our spatiotemporal fusion feature from STResNet appearance model is incorporated into the correlation filter for robust visual object tracking. The experimental results show that our method achieves similar or better performance compared with the other tracking methods based on convolutional neural networks or correlation filter.

Fingerprint orientation field (FOF) estimation plays a key role in enhancing the performance of the automated fingerprint identification system (AFIS): accurate estimation of FOF can evidently improve the performance of AFIS. However, despite the enormous attention on the FOF estimation research in the past decades, the accurate estimation of FOFs, especially for poor-quality fingerprints, still remains a challenging task. In this paper, we devote to review and categorization of the large number of FOF estimation methods proposed in the specialized literature, with particular attention to the most recent work in this area. Broadly speaking, the existing FOF estimation methods can be grouped into three categories: gradient-based methods, mathematical models-based methods, and learning-based methods. Identifying and explaining the advantages and limitations of these FOF estimation methods is of fundamental importance for fingerprint identification, because only a full understanding of the nature of these methods can shed light on the most essential issues for FOF estimation. In this paper, we make a comprehensive discussion and analysis of these methods concerning their advantages and limitations. We have also conducted experiments using publically available competition dataset to effectively compare the performance of the most relevant algorithms and methods.

Alternaria leaf spot, Brown spot, Mosaic, Grey spot, and Rust are five common types of apple leaf diseases that severely affect apple yield. However, the existing research lacks an accurate and fast detector of apple diseases for ensuring the healthy development of the apple industry. This paper proposes a deep learning approach that is based on improved convolutional neural networks (CNNs) for the real-time detection of apple leaf diseases. In this paper, the apple leaf disease dataset (ALDD), which is composed of laboratory images and complex images under real field conditions, is first constructed via data augmentation and image annotation technologies. Based on this, a new apple leaf disease detection model that uses deep-CNNs is proposed by introducing the GoogLeNet Inception structure and Rainbow concatenation. Finally, under the hold-out testing dataset, using a dataset of 26,377 images of diseased apple leaves, the proposed INAR-SSD (SSD with Inception module and Rainbow concatenation) model is trained to detect these five common apple leaf diseases. The experimental results show that the INAR-SSD model realizes a detection performance of 78.80% mAP on ALDD, with a high-detection speed of 23.13 FPS. The results demonstrate that the novel INAR-SSD model provides a high-performance solution for the early diagnosis of apple leaf diseases that can perform real-time detection of these diseases with higher accuracy and faster detection speed than previous methods.

In Japan, blue and chub mackerels are often caught simultaneously, and their market prices are different. Humans need to sort them manually, which requires heavy labor. The demand for automatic sorting machines is increasing. The aim of this paper is to develop an automatic sorting machine of mackerels, which is a challenging task. There are two required functions. First, it needs localization of mackerels on a conveyor belt so that mackerels can be transported to destinations. Second, species classification is needed, but it is difficult due to similar appearance among the species. In this paper, we propose an automatic sorting machine using deep neural networks and a red laser light. Specifically, we irradiate red laser to the abdomen, and the shape of the laser will be circle and ellipse on the blue and chub mackerels, respectively. We take images and use neural networks to locate the whole body and irradiated regions. Then, we classify mackerels using features extracted from the whole body and irradiated regions. Using both features makes the classification accurate and robust. The experimental results show that the proposed classification is superior to the methods using either feature of irradiated or whole body regions. Moreover, we confirmed that the automatic mackerel-sorting machine performs accurately.

This paper proposes an improved image high fidelity compression algorithm based on the generative adversarial networks (GANs) to deal with the problem that the UAV image has a large amount of data which is not conducive to post-processing. By adding an encoder in front of the generator, the disaster area image transmitted by UAV is compressed to meet the requirements of the generator. After the compressed image is trained together with the real image through the discriminator, the quality of the compressed image is constantly improved. This image compression algorithm can fully synthesize the codes of non-major areas such as trees and rivers in the image, and try to retain the codes of important areas such as houses and roads. The experimental results show that the proposed compression method in this paper has a higher compression ratio than the traditional compression method for the disaster area image, and can obtain images with strong sense of hierarchy.

In this paper, we introduce the DEEP-HEAR framework, a multimodal dynamic subtitle positioning system designed to increase the accessibility of deaf and hearing impaired people (HIP) to multimedia documents. The proposed system exploits both computer vision algorithms and deep convolutional neural networks specifically designed and tuned in order to detect and recognize the identity of the active speaker. The main contributions of the paper concern: a novel method dedicated to recognizing various characters existent in the video stream. A video temporal segmentation algorithm that divides the video sequence into semantic units, based on face tracks and visual consistency. Finally, the core of our approach concerns a novel active speaker recognition method relying on the multimodal information fusion from the text, audio, and video streams. The experimental results carried out on a large scale dataset of more than 30 videos, validate the proposed methodology with average accuracy and recognition rates superior to 90%. Moreover, the method shows robustness to important object/camera motion and face pose variation, yielding gains of more than 8% in precision and recall rates when compared with state-of-the-art techniques. The subjective evaluation of the proposed dynamic subtitle positioning system demonstrates the effectiveness of our approach.

Optical coherence tomography (OCT) technology enables experts to analyze coronary lesions from high-resolution intravascular images. Studies have shown a relationship between vascular bifurcation and a higher occurrence of wall thickening and lesions in these areas. Some level of automation could benefit experts since the visual analysis of pullback frames is a laborious and time-consuming task. Although convolutional neural networks (CNNs) have shown promising results in classifying medical images, in this paper, we found no studies using CNNs in IVOCT images to classify the vascular bifurcation. In this paper, we evaluated four different CNN architectures in the bifurcation classification task trained with the IVOCT images from nine pullbacks from nine different patients. We used data augmentation to balance the dataset, due to the small number of bifurcation-labeled frames, and also applied transfer learning methods to incorporate the knowledge from a lumen segmentation task into some of the evaluated networks. Our classification outperforms other works in this literature, presenting AUC = 99.72%, obtained by a CNN with transferred knowledge.

Accurate and timely prediction of remaining useful life (RUL) of a machine enables the machine to have an appropriate operation and maintenance decision. Data-driven RUL prediction methods are more attractive to researchers because they can be deployed quicker and cheaper compared to other approaches. The existing deep neural network (DNN) models proposed for the applications of RUL prediction are mostly single-path and top-down propagation. In order to improve the prognostic accuracy of the network, this paper proposes a directed acyclic graph (DAG) network that combines long short term memory (LSTM) and a convolutional neural network (CNN) to predict the RUL. Different from the existing prediction models combined with CNN and LSTM, the method proposed in this paper combines CNN and LSTM organically instead of just using CNN for feature extraction. Moreover, when a single timestamp is used as an input, padding the signals in the same training batch would affect the prediction ability of the developed model. To overcome this drawback, the proposed method generates a short-term sequence by sliding the time window (TW) with one step size. In addition, based on the degradation mechanism, the piece-wise RUL function is used instead of the traditional linear function. In the experimental test, the turbofan engine degradation simulation dataset provided by NASA is used to validate the proposed RUL prediction model. By comparing with the existing methods using the same dataset, it can be concluded that the prediction method proposed in this paper has better prediction capability.

This paper presents a comprehensive review of the principle and application of deep learning in retinal image analysis. Many eye diseases often lead to blindness in the absence of proper clinical diagnosis and medical treatment. For example, diabetic retinopathy (DR) is one such disease in which the retinal blood vessels of human eyes are damaged. The ophthalmologists diagnose DR based on their professional knowledge, that is labor intensive. With the advances in image processing and artificial intelligence, computer vision-based techniques have been applied rapidly and widely in the field of medical images analysis and are becoming a better way to advance ophthalmology in practice. Such approaches utilize accurate visual analysis to identify the abnormality of blood vessels with improved performance over manual procedures. More recently, machine learning, in particular, deep learning, has been successfully implemented in this area. In this paper, we focus on recent advances in deep learning methods for retinal image analysis. We review the related publications since 1982, which include more than 80 papers for retinal vessels detections in the research scope spanning from segmentation to classification. Although deep learning has been successfully implemented in other areas, we found only 17 papers so far focus on retinal blood vessel segmentation. This paper characterizes each deep learning based segmentation method as described in the literature. Analyzing along with the limitations and advantages of each method. In the end, we offer some recommendations for future improvement for retinal image analysis.

Edge computing aims to integrate computing into everyday settings, enabling the system to be context-aware and private to the user. With the increasing success and popularity of deep learning methods, there is an increased demand to leverage these techniques in mobile and wearable computing scenarios. In this paper, we present an assessment of a deep human activity recognition system's memory and execution time requirements, when implemented on a mid-range smartphone class hardware and the memory implications for embedded hardware. This paper presents the design of a convolutional neural network (CNN) in the context of human activity recognition scenario. Here, layers of CNN automate the feature learning and the influence of various hyper-parameters such as the number of filters and filter size on the performance of CNN. The proposed CNN showed increased robustness with better capability of detecting activities with temporal dependence compared to models using statistical machine learning techniques. The model obtained an accuracy of 96.4% in a five-class static and dynamic activity recognition scenario. We calculated the proposed model memory consumption and execution time requirements needed for using it on a mid-range smartphone. Per-channel quantization of weights and per-layer quantization of activation to 8-bits of precision post-training produces classification accuracy within 2% of floating-point networks for dense, convolutional neural network architecture. Almost all the size and execution time reduction in the optimized model was achieved due to weight quantization. We achieved more than four times reduction in model size when optimized to 8-bit, which ensured a feasible model capable of fast on-device inference.

The time-frequency analysis of vibration signals is an effective means to analyze the fault characteristics of rolling bearings. The traditional pattern recognition method is difficult to adapt to the complex mapping relationship between the high-dimensional feature space and the state space. The deep learning method has high-dimensional feature adaptive analysis ability, which is suitable for the intelligent analysis of the high-dimensional feature space in fault states. The feedforward deep convolutional neural network (CNN) has achieved some success in mechanical fault diagnosis. However, the rolling bearing fault signal is complex, and there are many interference factors. The CNN relying on the simple feedforward method cannot effectively meet the actual needs in the field of fault diagnosis. Although there are some CNNs with feedback methods, the CNNs of these feedback methods cannot systematically obtain the characteristic information of rolling bearing faults. Therefore, they do not solve the feature extraction problem of rolling bearing faults well. In view of this, this paper provides a specific mathematical definition of the feedback mechanism for constructing the feedback mechanism in the deep CNN, models the feedback mechanism into an optimization problem, determines the basic framework of the feedback mechanism, and an effective feedback mechanism calculation model is proposed. Based on this, a solution algorithm based on the gradient descent method is proposed. Then, an effective supervised feature extraction method based on sparse expression is proposed. It maps the sample features to the feature domain through the effective transform method. In the process, the wavelet packet transform (WPT) transform is used as the basis function to construct a dictionary with structural effects, and mixed penalty terms are introduced to further optimize the performance of structural sparse expression. Finally, the sparse expression is combined with the feedback mechanism CNN (FCNN) to establish a sub-module fault diagnosis network so that a diagnosis can determine the fault severity while assessing the bearing fault location. The example shows that the method proposed in this paper has high accuracy in determining the state of rolling bearings and has great application potential in engineering.

In video analysis, collection and labeling of data can be time and resource-consuming. To solve the scarcity of data problems, synthetic data augmentation is a promising solution. In this paper, we present an approach to generate synthetic videos for action recognition using Unity, the popular game engine. The synthetic videos are generated with high variability in lighting, subjects’ models, backgrounds, animations, and camera positions. We use the generated data to augment a small dataset of subjects who are executing physical exercises for action recognition. We tested the augmented data on two state-of-the-art models for action classification and demonstrated the significant benefits of synthetic data augmentation for improving the performance of these models on small datasets in the context of video action recognition.

The gross domestic product (GDP) is the most widely used indicator in macroeconomics and the main tool for measuring a country’s economic output. Due to the diversity and complexity of the world economy, a wide range of models have been used, but there are challenges in making decadal GDP forecasts given unexpected changes such as emergence of catastrophic world events including pandemics and wars. Deep learning models are well suited for modelling temporal sequences and time series forecasting. In this paper, we develop a deep learning framework to forecast the GDP growth rate of the world economy over a decade. We use the Penn World Table as the data source featuring 13 countries prior to the COVID-19 pandemic, such as Australia, China, India, and the United States. We present a recursive deep learning framework to predict the GDP growth rate in the next ten years. We test prominent deep learning models and compare their results with traditional econometric models for selected developed and developing countries. Our decadal forecasts reveal that that most of the developed countries would experience economic growth slowdown, stagnation and even recession within five years (2020-2024). Furthermore, our model forecasts show that only China, France, and India would experience stable GDP growth.

Automated analysis of lung sounds is a non-invasive technique that has the potential to become a powerful tool for the detection of respiratory illnesses. In this paper, we propose a machine learning method tasked to predict the number of respiratory crackles in a recording. More specifically, a Convolutional Neural Network (CNN) regression algorithm with a spectrogram input. Firstly, a large set of simulated data is used to train four different CNN designs. We use a lung sound simulator because of the absence of a large public database with sufficiently labelled lung recordings. When comparing the performance of these designs on simulated sounds, we concluded that maximum-pooling performed better than stride in the convolution layers. Furthermore, multiple consecutive convolution layers with small kernels outperformed single convolution layers using large kernels with the same receptive field. The best performing CNN achieved a root mean square error of 6.26 crackles per recording with an average of 50 crackles per recording. Secondly, we validate the best CNN design on a set of 51 real respiratory recordings that are labelled with a perceived degree of crackles: no crackles, mild, moderate, and severe. The system is capable of detecting an increasing amount of crackles in the respective last three categories. These results show the effectiveness of this novel approach to use simulated data for the development of lung sound analysis algorithms.

Mel-Frequency Cepstral Coefficients (MFCCs) have been extensively used as input for many traditional and modern speech processing systems. The power of MFCCs lies in the compact representation of speech signals, which is capable of capturing the essential phonetic content of the speech. However, most of the MFCC energy concentrates on the low-order coefficients, and the flat distribution of high-order MFCC values makes convolutional operators less sensitive to the transient details of the coefficients, which may be important in certain speech processing tasks like speaker recognition. In this paper, we analyze the differences between Mel-scale filter bank energies (MFBEs) and MFCCs, and we show that MFBEs are more effective inputs for deep learning-based Vietnamese speaker verification. MFBEs help deep learning models learn a better speaker representation with a more compact distribution of embedding vectors. Experiments on two Vietnamese speaker verification datasets show that the MFBEs consistently outperform MFCCs in improving the performance of some state-of-the-art deep learning models. The equal error rate (EER) on the Vietnam-Celeb test dataset was reduced by 1.14% with the ResNetSE-34 model and 2.36%, or 51.6% improvement, on the VLSP2021 test dataset with ECAPA-TDNN model and transfer learning.

This paper introduces a robust multiplatform artificial intelligence (AI) system designed for real-time emotion classification in service robotics. The primary focus of this study is the development and implementation of an initial phase AI model within a diverse hardware ecosystem, enabling a Service Robot to interact dynamically with users in real-time, complemented by cloud connectivity and enhanced by fog computing techniques. Utilizing convolutional neural networks (CNNs) via TensorFlow in Python, emotion classification models were crafted to discern various emotional states accurately. The AI system’s multiplatform environment comprised three distinct processing units: a laptop with Windows OS and high-end specifications, Jetson Orin running on a Linux architecture, and Raspberry Pi-4 operating on Raspbian OS. The inclusion of fog computing allowed for local data processing near the source, significantly reducing latency and ensuring faster response times, crucial for real-time applications. Training and comparative analysis were conducted on both the laptop and Jetson Orin, leveraging the GPU capabilities of each device. The Raspberry Pi-4 was allocated for monitoring and sensor data collection. Results revealed an impressive average performance of 90% in accurately classifying diverse emotional states. Training duration ranged from 20 minutes on the laptop to 30 minutes on Jetson Orin, with faster program execution observed on Jetson Orin due to its lightweight operating system. Cloud connectivity played a pivotal role in wirelessly monitoring, controlling, and facilitating real-time data collection. Additionally, it enabled the implementation of alert systems and notifications throughout the classification process. This pioneering approach lays a robust foundation for the advancement of sophisticated human-machine interaction systems, setting the stage for further research in the realm of AI-driven service robotics.

The rapid evaluation of smart cities has revolutionized the research and development field to a very extensive level which presents challenges in handling massive amounts of data. However, the integration of IoT into various aspects of life has introduced various challenges related to the security and privacy of IoT systems. IoT sensors capture large volumes of sensitive customer data, which can potentially make them a target and pose serious threats, including financial loss and identity theft. Strong intrusion detection systems are essential for protecting networked, data-driven ecosystems from potential cyber threats. In this paper, we propose a novel deep learning-based approach that focuses on emerging zero-touch networks that autonomously manage network resources to ensure network security, the proposed approach identifies various network intrusions such as DDoS, Botnet, Brute force, and Infiltration. Our proposed approach presents a major improvement in IoT security. We have used the CICIDS-2018 benchmark dataset and propose a deep learning-based network intrusion detection System for Zero Touch Networks (DL-NIDS-ZTN). The proposed study utilizes convolutional neural networks that correctly identify benign and malicious traffic and achieve 99.80% accuracy with the CICIDS-2018 dataset. By implementing the DL-NIDS-ZTN methodology, we aim to strengthen the security framework of smart cities and ensure the secure and seamless integration of IoT.

The growing use of deep learning models in end-device applications has led to various inference attacks and associated data privacy threats. Recent research also reveals the susceptibility of CNN models to model inversion-inference attacks, including the reversal of private training data. In response, this paper proposes a novel defense strategy called SecureLite, which defends against model inversion attacks through a series of complex data processing and data augmentation steps, followed by a novel adversarial training-aware model obfuscation (ATMO) algorithm. The ATMO involves adversarial training with defensive distillation, coupled with Laplace weight obfuscation. Our results demonstrate that SecureLite effectively mitigates model inversion attacks. This is evidenced by image classification tasks on four different CNN target models, including Simple CNN, AlexNet, VGG-16, and MobileNet-V1, resulting in reductions ranging from 85.69% to 99.36% in the ASR for model inversion attacks tested on multiple datasets, including CIFAR-10, GTSRB, and Caltech-101. The inverted images are also evaluated using various benchmarking performance metrics, including LPIPS, SSIM, and PSNR, to ensure quality degradation. Furthermore, a comparative analysis with three other state-of-the-art defense mechanisms in the literature is conducted, revealing that our proposed model outperforms them, achieving superior results without compromising overall model performance. This method greatly enhances data privacy protection and model security, making a significant contribution to addressing unexpected issues and impacts in current technologies.

Automatic fault detection is currently combined with deep networks owing to the possibility of dispensing signal processing, which significantly accelerates reactions to faults. Changing the type of defect or object forces repetition of the network learning process and its implementation. The design of universal systems for detecting different faults can be developed using transfer learning techniques. This paper presents the application of the transfer learning of a convolutional neural network to the fault diagnosis of permanent magnet synchronous motors. The crucial point of this research was to develop accurate diagnostic applications based on the data obtained from the motor field model and to use their functionality for a real object. This study compares the accuracy of diagnostic systems using three currently known techniques: neural network-based, instance-based, and mapping-based transfer learning. The experimental verification of the systems was carried out on an experimental bench with a 2.5 kW motor.

In this paper, we propose a novel and enhanced approach for crowd counting within the domain of manatee monitoring, aiming to significantly improve efficiency and accuracy. The proposed model achieves state-of-the-art results in the challenging task of manatee counting, simplifying the work of scientists and experts in the field. Our model not only facilitates the identification and enumeration of manatees in images and videos but also excels in scenarios that pose considerable challenges for human observers. To enhance accurate counting of the manatee aggregation, we introduce a framework with three key innovations to tackle the challenge: a new approach to generate density maps during the training process, an augmented technique to balance the dataset, and a cross-domain solution to enhance overall performance. The proposed two-dimensional Gaussian kernel offers a refined method for creating density maps, providing a more robust foundation for the training phase. Additionally, we built a balanced and augmented dataset, ensuring that the model is exposed to diverse and representative instances, thus improving its generalization capabilities. Furthermore, we incorporate a cross-domain phase pretraining the model utilizing an image dataset of wild animals to initialize the weights and further improve performance. Experiments and comparisons, with respect to previously established CSRNET model presented in Wang et al. (2023), demonstrate noteworthy improvements. Remarkably, our model achieves a Mean Absolute Error (MAE) of nearly half compared to the rival approach, showcasing the substantial advancements achieved through our refined methodology. This progress boosts the reliability of manatee counting in conservation efforts and ecological research.

To improve the prediction accuracy of monocular depth estimation networks and address issues such as edge blurring and excessive artifacts in the generated depth maps, this paper proposes a deep network architecture based on a global-local feature fusion module and a residual pooling module. The encoder utilizes a Hierarchical Transformer, while the decoder incorporates a U-Net structure model that combines multi-dimensional attention features aggregation and residual pooling. The residual pooling module facilitates better extraction of background contextual information from the feature maps to obtain more precise scene depth information. The global-local feature fusion module enables the network to learn features that encompass both global and local information. Experimental evaluations conducted on the NYU Depth V2 and KITTI datasets demonstrate that the proposed method achieves a $\delta 1$ of 0.916 on the NYU Depth V2 dataset, along with enhanced generalization ability and robustness. Furthermore, the effectiveness of each module is validated through ablation studies on the NYU Depth V2 dataset.

Emotion recognition is a key research topic in the Affective Computing domain, with implications in marketing, human-robot interaction, and health domains. The continuous technological advances in terms of sensors and the rapid development of artificial intelligence technologies led to breakthroughs and improved the interpretation of human emotions. In this paper, we propose a lightweight neural network architecture that extracts and performs the analysis of multimodal information using the same audio and visual networks across multiple temporal segments. Undoubtedly, data collection and annotation for emotion recognition tasks remain challenging aspects in terms of required expertise and effort spent. In this sense, the learning process of the proposed multimodal architecture is based on an iterative procedure that starts with a small volume of annotated samples and allows a step-by-step improvement of the system by assessing the model uncertainty in recognizing discrete emotions. Specifically, at each epoch, the learning process is guided by the most uncertainly annotated samples and integrates different modes of expressing emotions through a simple augmentation technique. The framework is tested on two publicly available multimodal datasets for emotion recognition, i.e. CREMA-D and RAVDESS, using 5-folds cross-validation. Compared to state-of-the-art methods, the achieved performance demonstrates the effectiveness of the proposed approach, with an overall accuracy of 74.2 % on CREMA-D and 76.3 % on RAVDESS. Moreover, with a small number of model parameters and a low inference time, the proposed neural network architecture represents a valid candidate for the integration on platforms with limited memory and computational resources.