Search

• This is a comprehensive review of Transformer for Skeleton-based Action Recognition. • This paper proposes a new taxonomy of transformer-style techniques. • This survey aims to help researchers systematically select promising future directions. Skeleton-based action recognition has rapidly become one of the most popular and essential research topics in computer vision. The task is to analyze the characteristics of human joints and accurately classify their behaviors through deep learning technology. Skeleton provides numerous unique advantages over other data modalities, such as robustness, compactness, noise immunity, etc. In particular, the skeleton modality is extremely lightweight, which is especially beneficial for deep learning research in low-resource environments. Due to the non-European nature of skeleton data, Graph Convolution Network (GCN) has become mainstream in the past few years, leveraging the benefits of processing topological information. However, with the explosive development of transformer methods in natural language processing and computer vision, many works have applied transformer into the field of skeleton action recognition, breaking the accuracy monopoly of GCN. Therefore, we conduct a survey using transformer method for skeleton-based action recognition, forming of a taxonomy on existing works. This paper gives a comprehensive overview of the recent transformer techniques for skeleton action recognition, proposes a taxonomy of transformer-style techniques for action recognition, conducts a detailed study on benchmark datasets, compares the algorithm accuracy of standard methods, and finally discusses the future research directions and trends. To the best of our knowledge, this study is the first to describe skeleton-based action recognition techniques in the style of transformers and to suggest novel recognition taxonomies in a review. We are confident that Transformer-based action recognition technology will become mainstream in the near future, so this survey aims to help researchers systematically learn core tasks, select appropriate datasets, understand current challenges, and select promising future directions. [ABSTRACT FROM AUTHOR]

Automatic identity recognition from ear images is an active research topic in the biometric community. The ability to secretly acquire images of the ear remotely and the stability of the ear shape over time make this technology a promising alternative for surveillance, authentication, and forensic applications. In recent years, significant research has been conducted in this area. Nevertheless, challenges remain that limit the commercial use of this technology. Several phases of the ear recognition system have been studied in the literature, from ear detection, normalization, and feature extraction to classification. This paper reviews the most recent methods used to describe and classify biometric features of the ear. We propose a first taxonomy to group existing approaches to ear recognition, including 2D, 3D, and combined 2D and 3D methods, as well as an overview of historical advances in this field. It is well known that data and algorithms are the essential components in biometrics, particularly in-ear recognition. However, early ear recognition datasets were very limited and collected in laboratory with controlled environments. With the wider use of deep neural networks, a considerable amount of training data has become necessary if acceptable ear recognition performance is to be achieved. As a consequence, current ear recognition datasets have increased significantly in size. This paper gives an overview of the chronological evolution of ear recognition datasets and compares the performance of conventional vs. deep learning methods on several datasets. We proposed a second taxonomy to classify the existing databases, including 2D, 3D, and video ear datasets. Finally, some open challenges and trends are debated for future research. [ABSTRACT FROM AUTHOR]

The increasing availability of remote sensing data allows dealing with spatial-spectral limitations by means of pan-sharpening methods. However, fusing inter-sensor data poses important challenges, in terms of resolution differences, sensor-dependent deformations and ground-truth data availability, that demand more accurate pan-sharpening solutions. In response, this paper proposes a novel deep learning-based pan-sharpening model which is termed as the double-U network for self-supervised pan-sharpening (W-NetPan). In more details, the proposed architecture adopts an innovative W-shape that integrates two U-Net segments which sequentially work for spatially matching and fusing inter-sensor multi-modal data. In this way, a synergic effect is produced where the first segment resolves inter-sensor deviations while stimulating the second one to achieve a more accurate data fusion. Additionally, a joint loss formulation is proposed for effectively training the proposed model without external data supervision. The experimental comparison, conducted over four coupled Sentinel-2 and Sentinel-3 datasets, reveals the advantages of W-NetPan with respect to several of the most important state-of-the-art pan-sharpening methods available in the literature. The codes related to this paper will be available at https://github.com/rufernan/WNetPan. [ABSTRACT FROM AUTHOR]

• We focus on the data dilemma encountered in the field of human activity understanding, solve practical application problems from a new perspective, and use the proposed method to reduce the model's dependence on data. • We construct a human physical activity dataset containing 10 categories Human SA-10 for use in human activity understanding research. • A Core-Weight Entropy data information evaluation method that can be applied to human behavior recognition tasks is proposed. On Human SA-10, our method can reduce data usage by 50%. Compared to other methods, this method achieved state-of-the-art performance. • In addition, we propose a new unlabeled data redundancy information removal module, which effectively avoids introducing similar data into the training set. With the development of computer vision, the research on human activity understanding has been greatly promoted. The recognition algorithm based on vision transformer has made some achievements in a large number of computer vision tasks, but it still needs to be driven by a large amount of data. How to get rid of the constraints of large amounts of data is crucial for human behavior recognition based on vision transformer. This paper focuses on solving the dilemma of big data, and tries to achieve a high-performance model through a small amount of high information human activity data. The advantage of our work is that by studying feature distribution, we proposed a core weight entropy data information evaluation method for obtaining high information data, and through redundant information elimination strategy, we can avoid introducing similar data. A large number of experimental results show the effectiveness of the proposed method. Compared with existing methods, our method reduces the data consumption by 5% to 30%, and can achieve the performance of using only 50% of 100% data. More importantly, the data our method selected has no redundancy, which is not available in other methods. In addition, we carried out a large number of ablation experiments to prove the rationality of the method. The work of this paper solves the challenge of relying on a large amount of data when using the visual converter to recognize human behavior, which is of practical significance for realizing efficient human activity understanding research with low data. [ABSTRACT FROM AUTHOR]

[Display omitted] • The dense feature adaptive connection module improves the quality of depth map. • The distributed 3D convolution reduces the computational cost and memory space. • The joint loss function makes the network sensitive to small depth structures. The goal of Multi-View Stereo is to reconstruct the 3D point cloud model from multiple views. With the development of deep learning, more and more learning-based research has achieved remarkable results. However, existing methods ignore the fine-grained features of the bottom layer, which leads to the poor quality of model reconstruction, especially in terms of completeness. Besides, current methods still rely on a large amount of consumed memory resources because of the application of 3D convolution. To this end, this paper proposes a Multiple Granularities Feature Fusion Network for Multi-View Stereo, an end-to-end depth estimation network combining global and local features, which is characterized by fine-granularity multi-feature fusion. Firstly, we propose a dense feature adaptive connection module, which can adaptively fuse the global and local features in the scene, provide a more complete and effective feature map for inferring a more detailed depth map, and make the ultimate model more complete. Secondly, in order to further improve the accuracy and completeness of the reconstructed point cloud, we introduce normal and edge loss futead of only using depth loss functions as in the existing methods, which makes the network more sensitive to small depth structures. Finally, we propose distributed 3D convolution instead of traditional 3D convolution, which reduces memory consumption. The experimental results on the DTU and Tanks & Temples datasets demonstrate that the proposed method in this papaer achieves the state-of-the-art performance, which proves the accuracy and effectiveness of the MG-MVSNet proposed in this paper. [ABSTRACT FROM AUTHOR]

• Providing an in-depth and comprehensive review related to underwater object detection. • Extensively exploring the correlation between underwater image enhancement and underwater object detection. • Analyzing the impact and augmentation forms of underwater image enhancement in underwater object detection. • Discussing the challenges, future trends and applications of underwater object detection. Underwater object detection is one of the most challenging research topics in computer vision technology. The complex underwater environment makes underwater images suffer from high noise, low visibility, blurred edges, low contrast and color deviation, which brings significant challenges to underwater object detection tasks. In underwater object detection tasks, traditional object detection methods often perform poorly in terms of accuracy and generalization capabilities. Underwater object detection requires accurate, stable, generalizable, real-time and lightweight detection models, for which many researchers have proposed various underwater object detection techniques based on deep learning. Although many outstanding results have been achieved on underwater object detection over the years, the research status of underwater object detection techniques are still lack of unified induction, and some existing problems need to be further probed from the latest perspective. In addition, previous reviews lack analysis on the relationship between underwater image enhancement and object detection. Therefore, this paper provides a comprehensive review of the current research challenges, future development trends, and potential applications of underwater object detection techniques. More importantly, this paper has explored the internal relationship between underwater image enhancement and object detection, and analyzed the possible implementation manners of underwater image enhancement in the object detection task in order to further enhance its benefits. The experiments show the performances of current underwater image enhancement and state-of-the-art object detection algorithms, point out their limitations, and indicate that there is not a strict positive correlation between underwater image enhancement and the accuracy improvement of object detection. The domain shift caused by underwater image enhancement cannot be ignored. This paper can be regarded as a guide for future works on underwater object detection. [ABSTRACT FROM AUTHOR]

• A high-performance method is proposed for RGB-D salient object detection. • RGB and depth data are adaptively fused to boost the performance. • Features of all levels are refined in an iterative manner. • The proposed method outperforms other state-of-the-art models on six datasets. Existing state-of-the-art RGB-D saliency detection models mainly utilize the depth information as complementary cues to enhance the RGB information. However, depth maps can be easily influenced by environment and hence are full of noises. Thus, indiscriminately integrating multi-modality (i.e., RGB and depth) features may induce noise-degraded saliency maps. In this paper, we propose a novel Adaptive Fusion Network (AFNet) to solve this problem. Specifically, we design a triplet encoder network consisting of three subnetworks to process RGB, depth, and fused features, respectively. The three subnetworks are interlinked and form a grid net to facilitate mutual refinement of these multi-modality features. Moreover, we propose a Multi-modality Feature Interaction (MFI) module to exploit complementary cues between depth and RGB modalities and adaptively fuse the multi-modality features. Finally, we design the Cascaded Feature Interweaved Decoder (CFID) to exploit complementary information between multi-level features and refine them iteratively to achieve accurate saliency detection. Experimental results on six commonly used benchmark datasets verify that the proposed AFNet outperforms 20 state-of-the-art counterparts in terms of six widely adopted evaluation metrics. Source code will be publicly available at https://github.com/clelouch/AFNet upon paper acceptance. [ABSTRACT FROM AUTHOR]

• This paper introduces the mutual learning strategy train the brain tumor segmentation network, using the shallowest feature map to supervise the subsequent feature map of the network. using the deepest logits to supervise the previous shallow network's logits. The shallow feature map and deep logit supervise mutually and improve the accuracy of tumor sub-region segmentation. • This paper introduces the depth supervision to train this network, using the prediction of each up-sample layer is to deep supervise the training process for enlarging the receptive field to improve the overall segmentation accuracy. • A large number of experiments on BraTS dataset show that our method can effectively improve the accuracy of brain tumor segmentation and achieve the performance of SOTA. Deep learning methods have been successfully applied to Brain tumor segmentation. However, the extreme data imbalance exists in the different sub-regions of tumor, results in training the deep learning methods on these data will reduce the accuracy of segmentation. We introduce the deep mutual learning strategy to address the challenges, the proposed integrates transformer layers in both encoder and decoder of a U-Net architecture. In the network, using the prediction of up-sampled layer is to deep supervise the training process for enlarging the receptive field to extract features, the feature map of the shallowest layer supervises the subsequent feature map of layers to keep more edge information to guide the sub-region segmentation accuracy. the classification logits of the deepest layer supervise the previous layer of logits to get more semantic information for distinguish of tumor sub-regions. Furthermore, the feature map and the classification logits supervise mutually to improve the overall segmentation accuracy. The experimental results on benchmark dataset shows that our method has significant performance gain over existing methods. [ABSTRACT FROM AUTHOR]

• This paper proposes a filter pruning approach for CNN model compression by transferring the knowledge of unimportant filters to the filters of higher importance. • Before pruning unimportant filters, a custom regularizer is utilized for knowledge transfer, which increases the gap between L1-norms of important and unimportant filters. • The effect of the penalty imposed in the custom regularizer is analyzed to justify the need for knowledge transfer before pruning. • In order to validate the robustness of the proposed framework across different CNN architectures, we experiment with five popular CNNs, namely, LeNet-5, VGG-16, ResNet-56, ResNet-110, and ResNet-50. • Experiments are performed on three benchmark datasets MNIST, CIFAR-10, and ImageNet. • An improvement over the baseline in terms of accuracy is observed even after removing 95.15%, 62.28%, and 62.39% of the Floating Point OPerations (FLOPs) from architectures LeNet-5, ResNet-56, and ResNet-110, respectively. As the deep learning models have been widely used in recent years, there is a high demand for reducing the model size in terms of memory and computation without much compromise in the model performance. Filter pruning is a very widely adopted strategy for model compression. The existing filter pruning methods identify the unimportant filters and prune them without worrying about information loss. They try to recover the same by fine-tuning the remaining filters, limiting their performance. In this paper, we tackle this problem by utilizing the knowledge from unimportant filters before pruning to minimize information loss. First, the proposed method identifies the unimportant and important filters by exploiting the lower and higher importance, respectively, using the L 1 -norm of filters. Next, the proposed custom UFKT-Reg regularizer ( R ufkt ) transfers the knowledge from unimportant filters before pruning to remaining filters, notably to a fixed number of important filters. Hence, the proposed method minimizes information loss due to the removal of unimportant filters. The experiments are conducted using the three benchmark datasets, including MNIST, CIFAR-10, and ImageNet. The proposed filter pruning method outperforms many recent state-of-the-art filter pruning methods. An improvement over the baseline in terms of accuracy is observed even after removing 95.15%, 62.28%, and 62.39% of the Floating Point OPerations (FLOPs) from architectures LeNet-5, ResNet-56, and ResNet-110, respectively. After pruning 53.25% of FLOPS from ResNet-50, only 1.02% and 0.47% of drops are observed in top-1 and top-5 accuracies, respectively. The code used in this paper will be publicly available at (https://github.com/sarvanichinthapalli/UFKT). [ABSTRACT FROM AUTHOR]

In this paper, an approach that makes use of knuckle creases and fingernails for person identification is presented. It introduces a framework for automatic person identification that includes localisation of the region of interest (ROI) of many components within hand images, recognition and segmentation of the detected components using bounding boxes, and similarity matching between two different sets of segmented images. The following hand components are considered: i) the metacarpophalangeal (MCP) joint, commonly known as the base knuckle; ii) the proximal interphalangeal (PIP) joint, commonly known as the major knuckle; iii) the distal interphalangeal (DIP) joint, commonly known as the minor knuckle; iv) the interphalangeal (IP) joint, commonly known as the thumb knuckle, and v) the fingernails. Crucial elements of the proposed framework are the feature extraction and similarity matching. This paper exploits different deep learning neural networks (DLNNs), which are essential in extracting discriminative high-level abstract features. We further use various similarity measures for the matching process. We validate the proposed approach on well-known benchmarks, including the 11k Hands dataset and the Hong Kong Polytechnic University Contactless Hand Dorsal Images known as PolyU. The results indicate that knuckle patterns and fingernails play a significant role in the person identification framework. The 11K Hands dataset results indicate that the left-hand results are better than the right-hand results and the fingernails produce consistently higher identification results than other hand components, with a rank-1 score of 100 %. In addition, the PolyU dataset attains 100 % in the fingernail of the thumb finger. [ABSTRACT FROM AUTHOR]

• The paper fills the gap in the literature by reviewing and analyzing the current studies that apply deep learning for Covid-19 forecasting. • The initial search identified 152 studies of which 53 passed the quality control. • The existing literature is categorized using a model-based taxonomy. • The description of the models along with their performance evaluation is presented. • Recommendations for future improvements are provided. The Covid-19 pandemic has galvanized scientists to apply machine learning methods to help combat the crisis. Despite the significant amount of research there exists no comprehensive survey devoted specifically to examining deep learning methods for Covid-19 forecasting. In this paper, we fill the gap in the literature by reviewing and analyzing the current studies that use deep learning for Covid-19 forecasting. In our review, all published papers and preprints, discoverable through Google Scholar, for the period from Apr 1, 2020 to Feb 20, 2022 which describe deep learning approaches to forecasting Covid-19 were considered. Our search identified 152 studies, of which 53 passed the initial quality screening and were included in our survey. We propose a model-based taxonomy to categorize the literature. We describe each model and highlight its performance. Finally, the deficiencies of the existing approaches are identified and the necessary improvements for future research are elucidated. The study provides a gateway for researchers who are interested in forecasting Covid-19 using deep learning. [ABSTRACT FROM AUTHOR]

Object detection is a basic vision task that accompanies people's daily lives all the time. The development of object detection technology has experienced an evolution from traditional-based algorithms to deep learning-based algorithms, which has made a qualitative leap in both detection accuracy and detection speed. With the advancement of deep learning, object detection techniques are increasingly becoming a part of everyday life, with the YOLO series of algorithms being extensively applied in various industries. In this paper, we initially present the frequently utilized datasets and evaluation criteria for object detection. Subsequently, we delve into the evolution of traditional object detection algorithms, highlighting two-stage and one-stage approaches through illustrative examples of classical methods. We also conduct a comprehensive summary and analysis of the detection results obtained by these methods. In addition, we introduce object detection applications in daily life, as well as the importance and some difficulties of these applications. Finally, we analyze and summarize the difficulties and challenges facing the task of object detection, and we look forward to the future development direction of object detection. • This survey is an extended version of our paper in ICIC2023. • We review the development and challenges of object detection. • We present the application of object detection and make prospects for the future. [ABSTRACT FROM AUTHOR]

With the popularization of computer technology, the number of malware has increased dramatically in recent years. Some malware can threaten the network security of users by downloading and installing, and even spreading widely on the Internet, causing consequences such as private data leakage in the operating system, extortion, and network paralysis. In order to deal with these threats, researchers analyze malicious samples through various analysis techniques, which are usually divided into static and dynamic analysis based on the principle of whether the code needs to be executed or not. This paper analyzes in detail several classical methods of feature extraction in malware detection techniques. With the technological development of artificial intelligence, deep learning is gradually being introduced into malware detection, which does not require the identification of professional security personnel and greatly improves the generalization ability of detection. In the paper, text-based detection methods, image visualization-based detection, and graph structure-based detection techniques are reviewed according to different feature extraction methods. In addition, the paper compares 26 datasets that have been commonly used in recent years applied in the research field and explains the main contents and specifications of the datasets. Finally, a summary and outlook of the malware research field is given. [ABSTRACT FROM AUTHOR]

Many deep neural network-based methods have recently been proposed for object detection due to the significant success of deep learning in computer vision. However, existing object detection methods typically extract the appearance features of objects from single image so that they usually suffer from poor performance in detecting micro Unmanned Aerial Vehicle (UAV), because micro UAV lacks of rich color, shape and texture information. To address this issue, we introduce the temporal information of objects from videos and develop a Spatio-Temporal Attention Module (STAM) to efficiently enhance feature map extraction for detecting micro UAV, and then integrate STAM into YOLOX to develop a video object detector for micro UAV. Meanwhile, we propose a lightweight Spatial Pyramid Pooling (SPP) module termed Group Simplified Spatial Pyramid Pooling-Fast with Cross Stage Partial (Group SimSPPFCSP) for the backbone's final stage layer to efficiently and lightly extract more semantic information, and we propose a neck with rich propagation pathways (NRPP) to facilitate the effective propagation of spatial and temporal information across different levels. Furthermore, we propose two data augmentation operations including SeqMosaic and SeqMixUp, to augment video data for video object detection. Experimental results show that our model can achieve competitive precision (with 5.0 mAP and 8.1 mAP S m a l l improvement) while maintaining real-time inference speed (35.3 fps). • This paper develops a spatio-temporal attention module which integrates spatial and temporal context information to enhance feature extraction of micro UAV. • This paper presents a lightweight SPP module named Group SimSPPFCSP to efficiently extract more semantic information and a neck with rich propagation pathways to facilitate the effective propagation of spatial and temporal information across different levels. Finally, the detection head is further optimized to focus on detecting small objects. • This paper proposes two data augmentation operations to improve the video detector for small objects. [ABSTRACT FROM AUTHOR]

Object detection using deep learning has attracted considerable interest from researchers because of its competency in performing state-of-the-art tasks, including detection, observation, and action recognition. Deep Learning (DL)-based object detection models extract features directly from data, which is more efficient and effective than traditional methods requiring handcrafted features. Further, DL also effectively tackles spatiotemporal challenges by leveraging techniques; hence, researchers can develop better object detection models and implement more efficient strategies for object recognition. Moreover, optimizing these models improves performance and recognizes objects within videos or images. This survey comprises of an overview of related review papers and DL-based Object Detection (OD) algorithms. Object detection algorithms are presented as two classifications, namely Two-stage and One-stage methods, with Convolutional Neural Network (CNN) as the backbone. OD's applications are examined here, and crowd analysis has been extensively studied and researched for potential applications. Most of the papers in object detection rely on Convolutional Neural Networks (CNNs) (28%), whereas crowd analysis papers are distributed as follows: 24% in counting, 25% in categorizing, and 25% in analyzing individual behaviors, and 27% in others. In recent years, deep learning has significantly advanced object detection capabilities to provide effective solutions for various applications, including crowd analysis. • GAI-Driven Crowd behavior analysis: The crowd behavior analysis in the future will integrate GAI with realtime monitoring to analyze extensive data from social media and surveillance, providing detailed insights into crowd sentiments and movements.This integration will enhance crowd management and reduce the risk of violence. • IoT and Edge: Future Trends in Crowd Management: Integration of IoT and edge computing in crowd management enables real-time data collection on crowd behavior, facilitating swift analysis to optimize resource allocation for enhanced safety and efficiency. • The Deepfakes positive impact in Crowd: In crowds, deepfakes such as DeepSync, and Deep-Media can be used positively to offer personalized entertainment experiences like interactive story-telling and virtual performances to foster community engagement and social interaction (Venkata Rao and Venugopal, 2024). • The Deepfakes negative impact in Crowd: In crowds, deepfakes can potentially damage community bonds and escalate social tensions by spreading misinformation and provoking conflicts among crowds. [ABSTRACT FROM AUTHOR]

This paper engages in a comprehensive investigation concerning the application of Explainable Artificial Intelligence (xAI) within the context of deep learning and Artificial Intelligence, with a specific focus on its implications for cybersecurity. Firstly, the paper gives an overview of xAI techniques and their significance and benefits when applied in cybersecurity. Subsequently, the authors methodically delineate their systematic mapping study, which serves as an investigative tool for discerning the potential trajectory of the field. This strategic methodological framework lets one identify the future research directions and opportunities that underlie the integration of xAI within the realm of Deep Learning, Artificial Intelligence, and cybersecurity, which are described in-depth. Then, the paper brings together all the gathered insights from this extensive investigation and closes with final conclusions. • Application of xAI in Network Intrusion Detection. • Research directions in integrating xAI with cybersecurity. • Guide for developing impactful xAI solutions. [ABSTRACT FROM AUTHOR]

• An overview of deep learning algorithms used in medical image segmentation is presented. • More than 150 papers applying deep learning to different medical applications are summarised. • Challenges and future directions in medical image segmentation are discussed. Deep learning (DL) algorithms have rapidly become a robust tool for analyzing medical images. They have been used extensively for medical image segmentation as the first and significant components of the diagnosis and treatment pipeline. Medical image segmentation is efficiently addressed by many types of deep neural networks, such as convolutional neural networks, fully convolutional network recurrent networks, adversarial networks, and U-shaped networks. This paper reviews the major DL models and applications pertinent to medical image segmentation and summarizes over 150 contributions to the field. Brief overviews of articles are provided by application area: anatomical structures such as organs, bones, and vessels, and abnormalities such as lesions and calcification. Moreover, we discuss current challenges and suggest directions for future research. [ABSTRACT FROM AUTHOR]

• A detailed literature review for the existing methods about anchor assignment and sample sampling is investigated and analyzed in a taxonomy. We identify a definition of each component, challenging problems in existing methods and their solution, and a comprehensive comparison of all methods. To the best of our knowledge, there is no prior work discussing these problems in object detection literature. • Open research direction in both anchor assignment and sampling heuristics is discussed. Based on these directions, we hope the object detection community can identify the current status of object detection methods to propose better solutions to anchor assignment and sampling heuristics. • We review the state-of-the-art object detection in the last two years that complement existing surveys. We also provide paper literature related to the above problems via the repository webpage. Deep learning-based object detection is a fundamental but challenging problem in computer vision field, has attracted a lot of study in recent years. State-of-the-art object detection methods rely on the selection of positive samples and negative samples, i.e., called sample assignment, and the definition of a useful set for training, i.e., called sample sampling heuristics. This paper presents a comprehensive review of the advanced anchor assignment and sampling approaches in deep learning-based object detection. Each problem is classified and analyzed systematically. According to the problem-based taxonomy, we identify the advantages and disadvantages of each problem in-depth and present open issues regarding the current methods. Furthermore, this paper also reviews the new trends in solving object detection that has not been discussed during the last two years. To track the latest research, a webpage related to the above problems is provided, which is available at https://github.com/VoXuanThuy/ObjectDetectionReview. [ABSTRACT FROM AUTHOR]

• Model-based and model-free gait recognition techniques apt for handling covariates. • An exploration of benefits/ limitations of deep learning pipelines under covariates. • List of covariates-based gait datasets for training robust gait recognition system. • Overview of the proposed fusion-based pipeline with the 3D dataset for covariates. • Research gaps and challenges in the implementation of deep learning pipelines. Gait recognition is an essential biometric technique that recognizes humans at a distance through their unique walking style. In the present era of deep learning, automated gait analysis has become easier with an increase in processing power. However, the recognition accuracy is affected by many covariates such as clothing conditions, carrying objects, varying viewing angles, occlusion, walking speed variations, and thus, it remains a challenging problem. For this complex problem, huge datasets are required to train for given conditions and predict new situations; thus, deep learning is preferred. In this review paper, we categorize various gait covariates which have been recently handled. There are various approaches, but the most effective approach is deep learning; hence in this paper, we include the most used deep learning approaches for each covariate condition found in the literature. Further, we highlight open problem areas handling these covariates and offer some suggestions about their better handling. Based on the review and our understanding of all the gait pipelines employed in deep learning, we have suggested a comprehensive and universal deep learning pipeline that can handle most gait covariates rather than customized deep learning pipelines. The methods of handling gait covariates are summarized according to appearance, pose, and sensors. A comprehensive comparison of reviewed approaches for real-time scenarios in terms of their novelty, benefits, and limitations is then offered, which led us to identify open research problems related to gait covariates. In the end, the paper concludes with the challenges identified and prospects. [ABSTRACT FROM AUTHOR]

Foreign species can deteriorate the environment and the economy of a country. To automatically monitor biosecurity threats at country borders, this paper investigates compact deep networks for accurate and real-time object segmentation for hyperspectral images. To this end, knowledge distillation (KD) approaches compress the model by distilling the knowledge of a large teacher network to a compact student network. However, when the student is over-compressed, the performance of standard KD methods degrades significantly due to the large capacity gap between the teacher and the student. This gap can be addressed by adding medium-sized teacher assistants, but training them incurs significant computation and hence is impractical. To address this problem, this paper proposes a new framework called Knowledge Distillation from Multi-head Teacher (KDM), which distills the knowledge of a multi-head teacher to the student. By encapsulating multiple teachers in a single network, our proposed KDM assists the learning of a very compact student and significantly reduces the training time. We also introduce Bio-HSI, a new large benchmark hyperspectral image dataset of 3,125 high-resolution images with dense segmentation ground truth. This new, large dataset can be expected to advance research on deep models for hyperspectral image segmentation. Evaluated on this dataset, the student trained via our KDM has 762 times fewer parameters than the state-of-the-art segmentation model (i.e., HRNet), while achieving competitive accuracy. [ABSTRACT FROM AUTHOR]

The problem of global warming is getting more serious, and vehicle emission is the main cause. In recent years, the number of locomotives in China has been increasing at a rate of more than 20% per year, and the problem of automobile pollution is becoming more serious. The transportation industry is the main source of fossil fuel combustion and environmental pollution. Therefore, in this paper, we propose a multi-view deep neural network (MVDNN) to analyze the key factors affecting the fuel consumption of automobiles. The experiments show that the introduction of human input improves the prediction accuracy and the root mean square error (RMSE) achieves 0.993. In addition, this paper also finds that for drivers, driving habits, driving frequency, and safety awareness are the most important factors affecting the fuel consumption of vehicles by combining Lasso regression with MVDNN. Finally, by comparing the prediction accuracy of different experiments, relevant policy suggestions are made. [ABSTRACT FROM AUTHOR]

Computational Ethology studies focused on human beings is usually referred as Human Activity Recognition (HAR). Specifically, this paper belongs to a line of work on the identification of broad cognitive activities that users carry out with computers. The keystone of this kind of systems is the noninvasive detection of the subject's gaze fixations in selected display areas. Noninvasiveness is ensured by using the conventional laptop cameras without additional illumination or tracking devices. The gaze ethograms, composed as sequences of gaze fixations, are the basis to identify the user activities. To determine the gaze fixation display areas with the highest accuracy, this paper explores the use of a transfer learning approach applied to several well-known deep learning network (DLN) architectures whose input is the eye area extracted from the face image,and output is the identification of the gaze fixation area in the computer screen. Two different datasets are created and used in the validation experiments. We report encouraging results that may allow the general use of the system. [ABSTRACT FROM AUTHOR]

The goal of semantic segmentation is to segment the input image according to semantic information and predict the semantic category of each pixel from a given label set. With the gradual intellectualization of modern life, more and more applications need to infer relevant semantic information from images for subsequent processing, such as augmented reality, autonomous driving, video surveillance, etc. This paper reviews the state-of-the-art technologies of semantic segmentation based on deep learning. Because semantic segmentation requires a large number of pixel-level annotations, in order to reduce the fine-grained requirements of annotation and reduce the economic and time cost of manual annotation, this paper studies the works on weakly-supervised semantic segmentation. In order to enhance the generalization ability and robustness of the segmentation model, this paper investigates the works on domain adaptation in semantic segmentation. Many types of sensors are usually equipped in some practical applications, such as autonomous driving and medical image analysis. In order to mine the association between multi-modal data and improve the accuracy of the segmentation model, this paper investigates the works based on multi-modal data fusion semantic segmentation. The real-time performance of the model needs to be considered in practical application. This paper analyzes the key factors affecting the real-time performance of the segmentation model and investigates the works on real-time semantic segmentation. Finally, this paper summarizes the challenges and promising research directions of semantic segmentation tasks based on deep learning. [ABSTRACT FROM AUTHOR]

The advent of deep learning has brought great change to the community of computer science and also revitalized numerous fields where traditional machine learning methods failed to make breakthroughs. Benefitted from the development of deep learning, analysis of medical images, which used to be a challenging yet exhausting task carried out manually by physicians, has experienced fast development as well. However, training deep learning models in these systems for analysis from scratch can be quite challenging. The small-scale data can't guarantee the performance of the developed systems, while large-scale data is usually unavailable due to expensive costs in the process of collection and storage. To allow a fast transition from one domain to another for reuse, experts and researchers have extensively delved transfer learning, which turns out to be an efficient and low-cost learning technique. In this paper, we will present a comprehensive survey of transfer learning on medical image analysis. The imaging modalities include but not limits to Computed Tomography (CT), Ultrasound (US), and Magnetic Resonance Imaging (MRI). The subjects covered in this paper include the brain, breast, lung, kidney, etc. Besides, this survey provides systematic knowledge about deep learning and transfer learning for beginners. Readers with different backgrounds can easily catch up with the interdisciplinary knowledge and new trends of transfer learning via this survey. [ABSTRACT FROM AUTHOR]

With the rapid development of society and the economy, autonomous driving techniques are widely applied in many areas, such as autonomous vehicles, autonomous drones, and robotics. As a dominating technique, deep learning has become more and more popular for 2-D and 3-D object detection. Numerous deep learning-based methods have been proposed to solve various vision issues. To further help with the development of unmanned systems, this paper presents a comprehensive survey of the recent processes from the past five years for 3-D object detection, road detection, traffic sign detection, and traffic light detection and classification. To summarize and analyze previous works in detail, this paper only focuses on deep learning-based object detection tasks in autonomous driving that take place when the input is a point cloud or image(s). It also presents comparative results for insight comparison and inspiring future researches. [ABSTRACT FROM AUTHOR]

Coronary artery disease is among one of the diseases human suffer most. Intravascular coronary arterial image analysis consists of denoising, segmentation, detection, and three-dimensional reconstruction, having a significant meaning for auxiliary diagnosis and treatment of coronary artery disease. Intravascular ultrasound (IVUS) and intravascular optical coherence tomography (IVOCT) are the two most commonly applied intravascular coronary arterial imaging techniques. Based on these fundamental imaging techniques, in recent years, many advanced technologies from traditional machine learning algorithms to deep learning methods were employed in the analysis of intravascular coronary arterial images and made huge progress in this field. In this survey, we reviewed more than one hundred papers published in top journals or conferences such as Neural Networks and MICCAI. These papers proposed approaches or schemes for the intravascular coronary arterial image analysis, including lumen border segmentation, atherosclerotic plaque characterization, media-adventitia segmentation, stent strut detection, and three-dimensional reconstruction. Our survey began with introducing coronary artery intravascular imaging techniques, essential neural networks, and deep learning and then presented an across-the-board review of methods, applications, and trends of intravascular image analysis. This survey is more comprehensive than other articles not only for its scope and reference number but also for discussing the future direction in this field. Compared to other review papers in this field, this article could assist beginners in constructing a basic knowledge frame of coronary artery intravascular image analysis methods and brought state-of-the-art progress in this field to fellow researchers. We hope this paper could benefit either the beginners for coronary arterial image analysis or experienced researchers. [ABSTRACT FROM AUTHOR]

Image recognition based on deep learning methods has gained remarkable achievements by feeding with abundant training data. Unfortunately, collecting a tremendous amount of annotated images is time-consuming and expensive, especially in grocery product recognition tasks. It is challenging to recognise grocery products accurately when the deep learning model is trained with insufficient data. This paper proposes multi-angle Generative Adversarial Networks (MAGAN), which can generate realistic training images with different angles for data augmentation. Mutual information is employed in the novel GAN to achieve the learning of angles in an unsupervised manner. This paper aims to create training images containing grocery products from different angles, thus improving grocery product recognition accuracy. We first enlarge the fruit dataset by using MAGAN and the state-of-the-art GAN variants. Then, we compare the top-1 accuracy results from CNN classifiers trained with different data augmentation methods. Finally, our experiments demonstrate that the MAGAN exceeds the existing GANs for grocery product recognition tasks, obtaining a significant increase in the accuracy. [ABSTRACT FROM AUTHOR]

In this paper, we present an extensive review on deep learning methods for image restoration tasks. Deep learning techniques, led by convolutional neural networks, have received a great deal of attention in almost all areas of image processing, especially in image classification. However, image restoration is a fundamental and challenging topic and plays significant roles in image processing, understanding and representation. It typically addresses image deblurring, denoising, dehazing and super-resolution. There are substantial differences in the approaches and mechanisms in deep learning methods for image restoration. Discriminative learning based methods are able to deal with issues of learning a restoration mapping function effectively, while optimisation models based methods can further enhance the performance with certain learning constraints. In this paper, we offer a comparative study of deep learning techniques in image denoising, deblurring, dehazing, and super-resolution, and summarise the principles involved in these tasks from various supervised deep network architectures, residual or skip connection and receptive field to unsupervised autoencoder mechanisms. Image quality criteria are also reviewed and their roles in image restoration are assessed. Based on our analysis, we further present an efficient network for deblurring and a couple of multi-objective training functions for super-resolution restoration tasks. The proposed methods are compared extensively with the state-of-the-art methods with both quantitative and qualitative analyses. Finally, we point out potential challenges and directions for future research. [ABSTRACT FROM AUTHOR]

Modern and new integrated technologies have changed the traditional systems by using more advanced machine learning, artificial intelligence methods, new generation standards, and smart and intelligent devices. The new integrated networks like the Internet of Things (IoT) and 5G standards offer various benefits and services. However, these networks have suffered from multiple object detection, localization, and classification issues. Conventional Neural Networks (CNN) and their variants have been adopted for object detection, classification, and localization in IoT networks to create autonomous devices to make decisions and perform tasks without human intervention and helpful to learn in-depth features. Motivated by these facts, this paper investigates existing object detection and recognition techniques by using CNN models used in IoT networks. This paper presents a Conventional Neural Networks for 5G-Enabled Internet of Things Network (CNN-5GIoT) model for moving and static objects in IoT networks after a detailed comparison. The proposed model is evaluated with existing models to check the accuracy of real-time tracking. The proposed model is more efficient for real-time object detection and recognition than conventional methods. [ABSTRACT FROM AUTHOR]

Studies, with the increasing concern for mental health, have shown that interventions along with social support can reduce stress and depression. However, counselling centers do not have enough resources to provide counselling and social support to all the participants in their interest. This paper helps social support organizations (e.g., university counselling centers) sequentially select the participants for interventions. Meanwhile, Deep Reinforcement Learning (DRL) has shown significant success in learning an efficient policy for sequential decision-making problems in both fully observable environments and partially observable environments with small action space. In this paper, we consider emotion propagation from other neighbours of the influencees, initial uncertainties of mental states and influence in the student network. We propose a new architecture called DRLPSO (Deep Reinforcement Learning with Particle Swarm Optimization) to enhance learning performance in a partially observable environment with a large state and action space. DRLPSO consists of two stages: the Discrete Particle Swarm Optimization (DPSO) and Deep Q-learning integrated with Long Short-Term Memory (DQ-LSTM). In the first stage, we apply DPSO by initializing n particles that converge to multiple optimal actions for each belief state. In the second stage, the action with the best Q-value from the DPSO action set is executed to obtain belief and observation (history of action). We evaluated the proposed method empirically with the simulated student networks with mental state propagation compared to the state-of-the-art algorithms. The experimental results demonstrate that DRLPSO outperforms the state-of-the-art DRL methods by an average of 32%. [ABSTRACT FROM AUTHOR]

Knowledge tracing presents a fundamental research challenge within personalized education, aiming to dynamically monitor students' evolving mastery of individual skills through analysis of their online answer data. Noteworthy advancements have been achieved in knowledge tracing models, particularly with the integration of deep learning techniques. These models leverage long-short-term memory (LSTM) networks to process students' answer sequences, discern latent skill states, and predict subsequent responses. Nevertheless, real-world scenarios introduce a substantial challenge, given the inherent sparsity of student interaction data, implying that each student interacts with only a limited number of skills. Deep knowledge tracing models, when confronted with sparse data, tend to exhibit a bias towards learning from students with large sample sizes, consequently resulting in compromised learning outcomes due to insufficient interaction data. Addressing these challenges, this paper introduces an innovative self-paced learning data augmentation approach designed to augment the number of student answer sequence samples. Additionally, the paper employs self-supervised contrastive learning to strike a balance between local and global information, thereby mitigating the model's inclination to overly focus on local samples. To effectively address the aforementioned challenges, the paper proposes a deep knowledge tracing framework grounded in self-paced contrastive learning. The efficacy of the self-paced contrastive learning strategy is validated using real datasets, demonstrating superior prediction accuracy compared to alternative algorithms. [Display omitted] • Self-paced data augmentation automatically discovers the difficulty of questions. • Self-paced data augmentation solves the problem of data sparseness in education. • Self-paced contrastive learning strategy improves the prediction accuracy of knowledge tracing models. [ABSTRACT FROM AUTHOR]

This paper introduces Elegans-AI models, a class of neural networks that leverage the connectome topology of the Caenorhabditis elegans to design deep and reservoir architectures. Utilizing deep learning models inspired by the connectome, this paper leverages the evolutionary selection process to consolidate the functional arrangement of biological neurons within their networks. The initial goal involves the conversion of natural connectomes into artificial representations. The second objective centers on embedding the complex circuitry topology of artificial connectomes into both deep learning and deep reservoir networks, highlighting their neural-dynamic short-term and long-term memory and learning capabilities. Lastly, our third objective aims to establish structural explainability by examining the heterophilic/homophilic properties within the connectome and their impact on learning capabilities. In our study, the Elegans-AI models demonstrate superior performance compared to similar models that utilize either randomly rewired artificial connectomes or simulated bio-plausible ones. Notably, these Elegans-AI models achieve a top-1 accuracy of 99.99% on both Cifar10 and Cifar100, and 99.84% on MNIST Unsup. They do this with significantly fewer learning parameters, particularly when reservoir configurations of the connectome are used. Our findings indicate a clear connection between bio-plausible network patterns, the small-world characteristic, and learning outcomes, emphasizing the significant role of evolutionary optimization in shaping the topology of artificial neural networks for improved learning performance. [ABSTRACT FROM AUTHOR]

Steganography and steganalysis are two interrelated aspects of the field of information security. Steganography seeks to conceal communications, whereas steganalysis aims to discover or, if possible, recover the data they contain. These two areas have garnered significant interest, especially among law enforcement agencies. Cybercriminals and even terrorists often employ steganography to avoid detection while in possession of incriminating evidence, even when that evidence is encrypted, since cryptography is prohibited or restricted in many countries. Therefore, a deep understanding of cutting-edge techniques for uncovering concealed information is essential in exposing illegal activities. Over the last few years, a number of strong and reliable steganography and steganalysis techniques have been introduced in the literature. This review paper provides a comprehensive overview of deep learning-based steganalysis techniques used to detect hidden information within digital media. The paper covers all types of cover in steganalysis, including image, audio, and video, and discusses the most commonly used deep learning techniques. In addition, the paper explores the use of more advanced deep learning techniques, such as deep transfer learning (DTL) and deep reinforcement learning (DRL), to enhance the performance of steganalysis systems. The paper provides a systematic review of recent research in the field, including data sets and evaluation metrics used in recent studies. It also presents a detailed analysis of DTL-based steganalysis approaches and their performance on different data sets. The review concludes with a discussion on the current state of deep learning-based steganalysis, challenges, and future research directions. • Detect covert messages through neural nets, enhancing steganalysis accuracy. • Steganalysis can benefit from auto-encoders, DNN, CNN, RNN, LSTM, GNN, DBN, and DRN. • Reinforcement learning, transfer learning, and adversarial-based methods are effective to steganalysis. • Transform domain, attention layer, and noise-based features empower steganalysis schemes. • Unsupervised learning extracts nuanced patterns from multimedia for detection. [ABSTRACT FROM AUTHOR]

In personal identity recognition systems, detecting a person's age, gender, and language using voice signal characteristics is a crucial issue, especially because of the importance of security considerations. Age, gender, and language classification problems are important in signal processing because they are used to analyze and understand human behavior, interactions, and preferences. This can be especially useful in the fields of human-computer interaction, psychology, and social science research. In this paper, a new system for detecting a speaker's age, gender, and language based on deep learning models is presented. Deep learning models have shown great efficacy in various fields of signal processing. For this paper, a range of deep models were tested, including convolutional neural networks (CNNs), recurrent neural network (RNN), and a fine-tuning ResNet34 architecture. Additionally, techniques such as transfer learning were applied to improve the efficiency of the proposed system. The input voice signals are preprocessed by applying the spectro-temporal transform to obtain additional features that can be fed to the ResNet34 model, which is designed specifically for the task of voice signal processing. The dataset used in this paper was sourced from the Mozilla common voice initiative, which is dedicated to advancing speech recognition and language identification technologies. The performance of the proposed algorithm was evaluated in the presence of Gaussian noise to determine its robustness. The experimental results demonstrated that the proposed algorithm significantly outperformed basic algorithms and other deep neural networks in terms of age and gender recognition from voice signals. [ABSTRACT FROM AUTHOR]

In March 2020, the World Health Organization (WHO) declared COVID-19 a global epidemic, caused by the SARS-CoV-2 virus. Initially, COVID-19 was diagnosed using real-time reverse transcription–polymerase chain reaction (RT-PCR) tests with a turnaround time of 2–3 days. To enhance diagnostic accuracy, medical professionals use medical imaging alongside RT-PCR. A positive result on both RT-PCR and medical imaging confirms a COVID-19 diagnosis. Imaging modalities like chest X-ray (CXR), computed tomography (CT) scans, and ultrasound are widely utilized for rapid and precise COVID-19 diagnoses. However, interpreting COVID-19 from these images is time-consuming and susceptible to human error. Therefore, leveraging artificial intelligence (AI) methods, particularly deep learning (DL) models, can deliver consistent, high-performance results. Unlike conventional machine learning (ML), DL models automate all stages of feature extraction, selection, and classification. This paper presents a comprehensive review of using DL techniques for diagnosing COVID-19 from medical imaging. The introduction provides an overview of diagnosing the coronavirus using medical imaging, highlighting associated challenges. Subsequently, the paper delves into key aspects of Computer-Aided Diagnosis Systems (CADS) based on DL methods for diagnosing COVID-19, covering segmentation, classification, explainable AI (XAI), and predictive research. Additionally, it reviews the rehabilitation systems such as the Internet of Medical Things (IoMT) in the context of COVID-19. In another section, uncertainty quantification (UQ) research is showcased, focusing on DL models for the diagnosis of Covid-19. Crucial challenges and future research directions are outlined in another section. Finally, discussion and conclusion sections are also provided at the end of the paper. [ABSTRACT FROM AUTHOR]

Deep learning (DL) has been shown to be very effective for many application domains of machine learning (ML), including image classification, voice recognition, natural language processing, and bioinformatics. The success of DL techniques is directly related to the availability of large amounts of training data. However, in many cases, the data are sensitive to the users and should be protected to preserve the privacy. Privacy-preserving deep learning (PPDL) has thus become a very active research field to ensure the training process and use of DL models are productive without exposing or leaking information about the data. This paper aims to provide a comprehensive survey of PPDL. We concentrate on the risks that affect data privacy in DL and conduct a detailed investigation into the models that ensure privacy. Finally, we propose a set of evaluation criteria, detailing the advantages and disadvantages of the solutions. Based on the analyzed strengths and weaknesses, the paper has highlighted some important research problems and application cases that have not been studied and these point to certain open research directions. [ABSTRACT FROM AUTHOR]

In this work, we examine the advantages of using multiple types of behaviours in recommendation systems. Intuitively, each user often takes some implicit actions (e.g., click) before making an explicit decision (e.g., purchase). Previous studies show that implicit and explicit feedback has different roles for a useful recommendation. However, these studies either exploit implicit and explicit behaviours separately or ignore the semantics of sequential interactions between users and items. In addition, we go from the hypothesis that a user's preferences at a time are combinations of long-term and short-term interests. In this paper, we propose some Deep Learning architectures. The first one is Implicit to Explicit (ITE) , to exploit users' interests through the sequence of their actions. The second and third ones are two versions of ITE with Bidirectional Encoder Representations from Transformers based (BERT-based) architecture called BERT-ITE and BERT-ITE-Si , which combine users' long- and short-term preferences without and with side information to enhance users' representations. The experimental results show that our models outperform previous state-of-the-art ones and also demonstrate our views on the effectiveness of exploiting the implicit to explicit order as well as combining long- and short-term preferences in three large-scale datasets. The source code of our paper is available at: https://github.com/tranquyenbk173/BERT_ITE. [ABSTRACT FROM AUTHOR]

• This paper presents a memory augmented convolutional neural network. • This paper utilizes the augmenting SOM as memory for better network generalization. • This paper instantiates the distributed knowledge representation with SOM and CNN. The long short-term memory (LSTM) network underpins many achievements and breakthroughs especially in natural language processing fields. Essentially, it is endowed with certain memory capabilities to boost its performance. Currently, the volume and speed of big data generation are increasing exponentially, and such data require efficient models to acquire memory augmented knowledge. In this paper, we propose a memory augmented convolutional neural network (MACNN) with utilizing self-organizing maps (SOM) as the memory module. First, we depict the potential challenge about just applying solely a convolutional neural network (CNN) so as to highlight the advantage of augmenting SOM memory for better network generalization. Then, we dissert a corresponding network architecture incorporating memory to instantiate the distributed knowledge representation machanism, which tactically combines both SOM and CNN. Each component of the input vector is connected with a neuron in a two-dimensional lattice. Finally, we test the proposed network on various datasets and the experimental results reveal that MACNN can achieve competitive performance, especially for bioimages datasets. Meanwhile, we further illustrate the learned representations to interpret the SOM behavior and to comprehend the achieved results, which indicates that the proposed memory-incorporating model can exhibit the better performance. [ABSTRACT FROM AUTHOR]

This paper focuses on two related sub-tasks of aspect-based sentiment analysis, namely aspect-term extraction and aspect sentiment classification. The former aims to extract aspect-terms from given sentences and the latter aims to identify the sentiment polarity expressed on the extracted terms. Considering the practical application, researchers use more joint methods rather than pipeline methods. However, existing joint methods cannot model the interaction between aspect-terms and the sentence they belong to, or consider the relevance among the sentiments of different aspect-terms. In this paper, a novel end-to-end generative model based on encoder-decoder, namely Joint Term-Sentiment Generator (JTSG), is presented to generate all aspect term-polarity pairs. Specifically, a pre-trained model based encoder is used to encode the sentences, and specially, the decoder generates the start and end position to determine an aspect-term, rather than generate aspect-terms themselves. This new generative method contributes to avoid generating incomplete aspect-terms. Experimental results demonstrate that the proposed approach yields competitive performance on three benchmark datasets. [ABSTRACT FROM AUTHOR]

The complex and varied urban road conditions have always been a difficult and a pivotal component in the study of driverless technology especially at intersections. In China, it is necessary for driverless cars to understand the gestures of traffic police. To identify the traffic police gesture at the intersection, the key step is to detect the traffic police at the intersection. At present, the research on traffic police detection is still in its infancy, there exists common problems such as slow detection speed and other real time problems in this method, and there is not a standardized traffic police data set ether. For the real time problems, this paper introduces the attention mechanism, and proposes a new real-time detection method of traffic police based on attention mechanism. The method proposed in this paper has strong robustness and can quickly complete the target detection task. For the data set problem, this paper analyzes and discloses similar data sets published through research. In the meanwhile, this paper collects 24,530 video data on the actual road, and it extracts and saves 12,000 pictures containing traffic police at frame rate of 30FPS as traffic police detection training and validated data set. [ABSTRACT FROM AUTHOR]

Single-object tracking is regarded as a challenging task in computer vision, especially in complex spatio-temporal contexts. The changes in the environment and object deformation make it difficult to track. In the last 10 years, the application of correlation filters and deep learning enhances the performance of trackers to a large extent. This paper summarizes single-object tracking algorithms based on correlation filters and deep learning. Firstly, we explain the definition of single-object tracking and analyze the components of general object tracking algorithms. Secondly, the single-object tracking algorithms proposed in the past decade are summarized according to different categories. Finally, this paper summarizes the achievements and problems of existing algorithms by analyzing experimental results and discusses the development trends. [ABSTRACT FROM AUTHOR]

The purpose of this paper is to give an overview of various well known medical image registration techniques with a special focus on registration between anatomical and functional medical images. Examples of anatomical medical images (AMI) are Computer Tomography (CT), Magnetic Resonance Images (MRI), X-ray radiographs and ultrasound etc. whereas Positron Emission Tomography (PET), Single Photon Emission Tomography (SPECT) and fMRI are examples of functional medical images (FMI). Irrespective of such types (AMI or FMI), every case can be considered as a medical imaging modality. All these modalities are widely used by clinicians to study the structure/functionality of human body parts for diagnosis and treatment. It is frequently required to combine PET/SPECT with CT/MRI to simultaneous study the metabolic and molecular information received through PET/SPECT with fine anatomical details observed by CT/MRI. This concurrent study will help in the diagnosis and localization of many diseases like cancer, blockage in coronary arteries and brain-related diseases like Parkinson, Alzheimer etc. Further, in many cases, clinicians are required to co-register one or more anatomical images with functional images, for example, ultrasound-guided biopsy fused with PET and MRI. This registration can be done either at the hardware level or at the software level. The introduction of integrated PET-CT machine increases the acceptability of hardware-based registration systems as compared to software-based methods among the medics. One possible reason for this is the lack of validation of results achieved through software-based registration methods. On the other hand, software-based registration methods also have many advantages over the hardware-based registration systems like lesser exposure to radiation and no need for new investment on hardware etc. In this paper, both the methods with their merits and demerits are discussed in detail. [ABSTRACT FROM AUTHOR]

Deep Learning (DL) has been recently employed to build smart systems that perform incredibly well in a wide range of tasks, such as image recognition, machine translation, and self-driving cars. In several fields the considerable improvement in the computing hardware and the increasing need for big data analytics has boosted DL work. In recent years physiological signal processing has strongly benefited from deep learning. In general, there is an exponential increase in the number of studies concerning the processing of electromyographic (EMG) signals using DL methods. This phenomenon is mostly explained by the current limitation of myoelectric controlled prostheses as well as the recent release of large EMG recording datasets, e.g. Ninapro. Such a growing trend has inspired us to seek and review recent papers focusing on processing EMG signals using DL methods. Referring to the Scopus database, a systematic literature search of papers published between January 2014 and March 2019 was carried out, and sixty-five papers were chosen for review after a full text analysis. The bibliometric research revealed that the reviewed papers can be grouped in four main categories according to the final application of the EMG signal analysis: Hand Gesture Classification, Speech and Emotion Classification, Sleep Stage Classification and Other Applications. The review process also confirmed the increasing trend in terms of published papers, the number of papers published in 2018 is indeed four times the amount of papers published the year before. As expected, most of the analyzed papers (≈ 60 %) concern the identification of hand gestures, thus supporting our hypothesis. Finally, it is worth reporting that the convolutional neural network (CNN) is the most used topology among the several involved DL architectures, in fact, the sixty percent approximately of the reviewed articles consider a CNN. [ABSTRACT FROM AUTHOR]

Many methods achieve the visual object tracking task with deep learning technologies. As the deep features of different levels contain various semantic information and functions, this paper presents a multi-hierarchy feature aggregation approach to tackle the specific issues in the tracking task, which consists of two aspects. On one hand, this paper integrates the features captured by the offline and online classifiers at the score level, which constructs complementary roles of these classifiers to enhance the stability of classification. Besides, the proposed offline classifier is continuously optimized with different levels of features to reinforce classification constraints. On the other hand, we design a butterfly attention module to promote the capacity of multi-hierarchy feature aggregation in the regression network, which aims to fuse and strengthen the multi-scale features by attending to their spatial information. It can capture more spatial contexts by utilizing the self-attention mechanism during the fusion procedure, and preserve the hierarchy of the features during the strengthening process. Extensive experiments on four public datasets, i.e., VOT2018, OTB100, NFS and LaSOT datasets, demonstrate the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

Deep learning has made substantial progress in crowd density estimation, but in practical application, due to the interference factors such as uneven distribution of crowd and changes in illumination, the existing methods still have large errors in counting. To solve the above problems, a crowd density estimation method based on multichannel dense grouping network is proposed. To solve the problem that it is difficult to extract the multiscale information of the crowd due to the uneven distribution of the crowd, multichannel dense grouping module (McDGM) is designed. In the module, Improved grouping convolution block (IGCB) is dense connected with other layers to obtain different levels of crowd characteristics, so as to avoid the loss of multi-scale information. In addition, the parameters of IGCB are reduced by group convolution. Then, to overcome the change of illumination, a crowd image enhancement algorithm is designed, which makes the image clear by the average pixel value and adjusting the contrast. Finally, to enhance the sensitivity of the network to crowd counting, a new loss function is proposed, which adds counting loss to the previous pixel space loss to improve the accuracy of crowd counting. The algorithm in this paper has been tested on ShanghaiTech and UCF_CC_50 datasets. The test results show that the algorithm in this paper has better statistical accuracy. [ABSTRACT FROM AUTHOR]

• A improved ELM-based auto-encoder is proposed for effective feature learning. • Low-rank matrix factorization is applied to learn optimal hidden features. • The proposed method lowers the random effect in feature learning of ELM-AE. • The proposed method enhances features representation ability of ELM-AE. • The proposed method achieves positive performances in feature learning. Extreme-learning-machine based auto-encoder (ELM-AE) is regarded as a useful architecture with fast learning speed and general approximation ability, and stacked ELM is used to develop efficient and effective deep learning networks. However, considering features learned from conventional ELM-AEs have issues of weak nonlinear representation ability and random factors in feature projection, this paper proposes an improved ELM-AE architecture which utilize low-rank matrix factorization to learn optimal low-dimensional features. Two superiorities can be obtained compared to conventional ELM-AEs. One is the dimensionality of the hidden layer in ELM-AE could be set arbitrarily, e.g. a higher-dimension hidden layer could lower the random effect in feature learning and enhance features representation ability. The other is enhancing features nonlinear ability, since features are learned directly from the nonlinear outputs of hidden layer. Finally, comparison experiments on numerical and image datasets are implemented in this paper to verify the superior performance of the proposed ELM-AE in this paper. [ABSTRACT FROM AUTHOR]

• This paper proposed a novel two-stage spatiotemporal features learning method based on a Siamese 3D CNN for MEs recognition. • The focal loss was introduced to address the problem of inefficient model training caused by the class imbalance in the MEs datasets. • An adaptive construction method based on adaptive CNN was proposed to construct the key-frames sequence to summarize the original MEs. Micro-expressions (MEs) are spontaneous and involuntary facial subtle reactions which often reveal the genuine emotions within human beings. Recognizing MEs automatically is becoming increasingly crucial for many areas such as diagnosis and security. However, the short time duration and low spatial intensity of MEs pose great challenges for accurately recognizing them. Additionally, the lack of sufficient and balanced spontaneous MEs data also makes this problem even harder to solve, and some adaptive modeling strategies have been quite urgent recently. To this end, this paper draws inspirations from few-shot learning to propose a novel two-stage learning (i.e., prior learning and target learning) method based on a siamese 3D convolutional neural network for MEs recognition (MERSiamC3D). Specifically, in the prior learning stage, the proposed MERSiamC3D is used to extract the generic features of MEs. In the target learning stage, the structure and parameters of the MERSiamC3D will be carefully adjusted and the Focal Loss is adopted for high-level features learning. Afterwards, in order to effectively retain the spatiotemporal information of the original MEs video, an adaptive construction method based on adaptive convolutional neural network is proposed to construct the key-frames sequence to summarize the original MEs video, which is able to help drop the redundant frames and relatively highlight the movement of the apex frame. Then, the new key-frames are taken as the input of the two-stage learning method. Finally, through extensive evaluations and experiments on three publically available MEs datasets, the proposed method in this work could outperform traditional methods and other deep learning baselines, which provides a novel insight on how to leverage scarce data for MEs recognition. [ABSTRACT FROM AUTHOR]

With the rapid development of deep learning technology and improvement in computing capability, deep learning has been widely used in the field of hyperspectral image (HSI) classification. In general, deep learning models often contain many trainable parameters and require a massive number of labeled samples to achieve optimal performance. However, in regard to HSI classification, a large number of labeled samples is generally difficult to acquire due to the difficulty and time-consuming nature of manual labeling. Therefore, many research works focus on building a deep learning model for HSI classification with few labeled samples. In this article, we concentrate on this topic and provide a systematic review of the relevant literature. Specifically, the contributions of this paper are twofold. First, the research progress of related methods is categorized according to the learning paradigm, including transfer learning, active learning and few-shot learning. Second, a number of experiments with various state-of-the-art approaches has been carried out, and the results are summarized to reveal the potential research directions. More importantly, it is notable that although there is a vast gap between deep learning models (that usually need sufficient labeled samples) and the HSI scenario with few labeled samples, the issues of small-sample sets can be well characterized by fusion of deep learning methods and related techniques, such as transfer learning and a lightweight model. For reproducibility, the source codes of the methods assessed in the paper can be found at https://github.com/ShuGuoJ/HSI-Classification.git. [ABSTRACT FROM AUTHOR]

Irregular time series data is becoming increasingly prevalent with the growth of multi-sensor systems as well as the continued use of unstructured manual data recording mechanisms. Irregular data and the resulting missing values severely limit the data's ability to be analysed and modelled for classification and forecasting tasks. Often, conventional methods used for handling time series data introduce bias and make strong assumptions on the underlying data generation process, which can lead to poor model predictions. Traditional machine learning and deep learning methods, although at the forefront of data modelling, are at best compromised by irregular time series data sets and fail to model the temporal irregularity of incomplete time series. Gated recurrent neural networks (RNN), such as LSTM and GRU, have had outstanding success in sequential modelling, and have been applied in many application fields, including natural language processing. These models have become an obvious choice for time series modelling and a promising tool for handling irregular time series data. RNNs have a unique ability to be adapted to make effective use of missing value patterns, time intervals and complex temporal dependencies in irregular univariate and multivariate time series data. In this paper, we provide a systematic review of recent studies in which gated recurrent neural networks have been successfully applied to irregular time series data for prediction tasks within several fields, including medical, human activity recognition, traffic monitoring and environmental monitoring. The review highlights the two common approaches for handling irregular time series data: missing value imputation at the data pre-processing stage and modification of algorithms to directly handle missing values in the learning process. Reviewed models are confined to those that can address issues with irregular time series data and does not cover the broader range of models that deal more generally with sequences and regular time series. This paper aims to present the most effective techniques emerging within this branch of research as well as to identify remaining challenges, so that researchers may build upon this platform of work towards further novel techniques for handling irregular time series data. [ABSTRACT FROM AUTHOR]

• Recent development in deep learning for monocular depth estimation is reviewed. • Depth estimation is classified into supervised, unsupervised, and semi-supervised methods. • In terms of tasks, depth estimation is summarized as single-task and multi-task methods. Depth estimation is a classic task in computer vision, which is of great significance for many applications such as augmented reality, target tracking and autonomous driving. Traditional monocular depth estimation methods are based on depth cues for depth prediction with strict requirements, e.g. shape-from-focus/ defocus methods require low depth of field on the scenes and images. Recently, a large body of deep learning methods have been proposed and has shown great promise in handling the traditional ill-posed problem. This paper aims to review the state-of-the-art development in deep learning-based monocular depth estimation. We give an overview of published papers between 2014 and 2020 in terms of training manners and task types. We firstly summarize the deep learning models for monocular depth estimation. Secondly, we categorize various deep learning-based methods in monocular depth estimation. Thirdly, we introduce the publicly available dataset and the evaluation metrics. And we also analysis the properties of these methods and compare their performance. Finally, we highlight the challenges in order to inform the future research directions. [ABSTRACT FROM AUTHOR]