Search

In this paper, we propose a novel approach for face spoofing detection using a combination of color texture descriptors with a new convolutional neural network (CNN) architecture. The proposed approach is based on a new convolutional neural network architecture composed of two CNN parallel branches. The first branch is fed with complementary shallow local phase quantization (LPQ) invariant descriptors that result from joint color texture information from the hue, saturation, and value (HSV) color space to accurately capture the reflection properties of the face. Combining the HSV color space with LPQ is known to significantly improve performance. The second branch of the CNN takes an RGB image directly as input, effectively separating chromatic (color-related) information from achromatic (brightness-related) information in order to extract crucial facial color features. Each branch of the CNN produces a vector of deep features that are extracted. To effectively concatenate the deep features from the two output branches, we employ an attention mechanism based combination method. This method captures the complementarity of the two branches, improving the accuracy and robustness of the model. The combined feature vectors form an input vector for the next Dense layer, where the model can distinguish between live and spoofed faces. Our method detects 2D facial spoofing attacks involving printed photos and replayed videos. We showcase the effectiveness and superior performance of our approach through a series of experiments conducted on both the CASIA-FASD and Replay-Attack datasets. Our results are promising and surpassing those of other state-of-the-art methods on both used datasets in terms of 9 performance metrics. [ABSTRACT FROM AUTHOR]

Copy detection is a key task of image copyright protection. Most robust hashing schemes do not make satisfied performance of image copy detection yet. To address this, a robust hashing scheme with deep features and Meixner moments is proposed for image copy detection. In the proposed hashing, global deep features are extracted by applying tensor Singular Value Decomposition (t-SVD) to the three-order tensor constructed in the DWT domain of the feature maps calculated by the pre-trained VGG16. Since the feature maps in the DWT domain are slightly disturbed by digital operations, the constructed three-order tensor is stable and thus the desirable robustness is guaranteed. Moreover, since t-SVD can decompose a three-order tensor into multiple low-dimensional matrices reflecting intrinsic structure, the global deep feature calculation from the low-dimensional matrices can provide good discrimination. Local features are calculated by the block-based Meixner moments. As the Meixner moments are resistant to geometric transformation and can efficiently discriminate various images, the use of the block-based Meixner moments can make discriminative and robust local features. Hash is ultimately determined by quantifying and combining global deep features and local features. The results of extensive experiments on open image datasets demonstrate that the proposed robust hashing outperforms some state-of-the-art robust hashing schemes in terms of classification and copy detection performances. [ABSTRACT FROM AUTHOR]

Aggregating the diverse features into a compact representation is a hot issue in image retrieval. However, aggregating the differential feature of multilayer into a discriminative representation remains challenging. Inspired by the value-guided neural mechanisms, a novel representation method, namely, the multilayer feature aggregation histogram was proposed to image retrieval. It can aggregate multilayer features, such as low-, mid-, and high-layer features, into a discriminative yet compact representation via simulating the neural mechanisms that mediate the ability to make value-guided decisions. The highlights of the proposed method have the following: (1) A detail-attentive map was proposed to represent the aggregation of low- and mid-layer features. It can be well used to evaluate the distinguishable detail feature. (2) A simple yet straightforward aggregation method is proposed to re-evaluate the distinguishable high-layer feature. It can provide aggregated features including detail, object, and semantic by using semantic-attentive map. (3) A novel whitening method, namely difference whitening, is introduced to reduce dimensionality. It did not need to seek a training dataset of semantical similarity and can provide a compact yet discriminative representation. Experiments on the popular benchmark datasets demonstrate the proposed method can obviously increase retrieval performance in terms of mAP metric. The proposed method using 128-dimensionality representation can provide significantly higher mAPs than the DSFH, DWDF, and OSAH methods by 0.083, 0.043, and 0.022 on the Oxford5k dataset and by 0.195, 0.036, and 0.071 on the Paris6k dataset. The difference whitening method can conveniently transfer the deep learning model to a new task. Our method provided competitive performance compared with the existing aggregation methods and can retrieve scene images with similar colors, objects, and semantics. [ABSTRACT FROM AUTHOR]

Contactless palmprint recognition offers friendly customer experience due to its ability to operate without touching the recognition device under rigid constrained conditions. Recent palmprint recognition methods have shown promising accuracy; however, there still exist some issues that need to be further studied such as the limited discrimination of the single feature and how to effectively fuse deep features and shallow features. In this paper, deep features and shallow features are integrated into a unified framework using feature‐level and score‐level fusion methods. Specifically, deep feature is extracted by residual neural network (ResNet), and shallow features are extracted by principal component analysis (PCA), linear discriminant analysis (LDA), and competitive coding (CompCode). In feature‐level fusion stage, ResNet feature and PCA feature are dimensionally reduced and fused by canonical correlation analysis technique to achieve the fused feature for the next stage. In score‐level fusion stage, score information is embedded in the fused feature, LDA feature, and CompCode feature to obtain a more reliable and robust recognition performance. The proposed method achieves competitive performance on Tongji dataset and demonstrates more satisfying generalization capabilities on IITD and CASIA datasets. Comprehensive validation across three palmprint datasets confirms the effectiveness of our proposed deep and shallow feature fusion approach. [ABSTRACT FROM AUTHOR]

Abstract Purpose 123I-Ioflupane SPECT is an effective tool for the diagnosis and progression assessment of Parkinson’s disease (PD). Radiomics and deep learning (DL) can be used to track and analyze the underlying image texture and features to predict the Hoehn-Yahr stages (HYS) of PD. In this study, we aim to predict HYS at year 0 and year 4 after the first diagnosis with combined imaging, radiomics and DL-based features using 123I-Ioflupane SPECT images at year 0. Methods In this study, 161 subjects from the Parkinson’s Progressive Marker Initiative database underwent baseline 3T MRI and 123I-Ioflupane SPECT, with HYS assessment at years 0 and 4 after first diagnosis. Conventional imaging features (IF) and radiomic features (RaF) for striatum uptakes were extracted from SPECT images using MRI- and SPECT-based (SPECT-V and SPECT-T) segmentations respectively. A 2D DenseNet was used to predict HYS of PD, and simultaneously generate deep features (DF). The random forest algorithm was applied to develop models based on DF, RaF, IF and combined features to predict HYS (stage 0, 1 and 2) at year 0 and (stage 0, 1 and ≥ 2) at year 4, respectively. Model predictive accuracy and receiver operating characteristic (ROC) analysis were assessed for various prediction models. Results For the diagnostic accuracy at year 0, DL (0.696) outperformed most models, except DF + IF in SPECT-V (0.704), significantly superior based on paired t-test. For year 4, accuracy of DF + RaF model in MRI-based method is the highest (0.835), significantly better than DF + IF, IF + RaF, RaF and IF models. And DL (0.820) surpassed models in both SPECT-based methods. The area under the ROC curve (AUC) highlighted DF + RaF model (0.854) in MRI-based method at year 0 and DF + RaF model (0.869) in SPECT-T method at year 4, outperforming DL models, respectively. And then, there was no significant differences between SPECT-based and MRI-based segmentation methods except for the imaging feature models. Conclusion The combination of radiomic and deep features enhances the prediction accuracy of PD HYS compared to only radiomics or DL. This suggests the potential for further advancements in predictive model performance for PD HYS at year 0 and year 4 after first diagnosis using 123I-Ioflupane SPECT images at year 0, thereby facilitating early diagnosis and treatment for PD patients. No significant difference was observed in radiomics results obtained between MRI- and SPECT-based striatum segmentations for radiomic and deep features.

Purpose: 123I-Ioflupane SPECT is an effective tool for the diagnosis and progression assessment of Parkinson's disease (PD). Radiomics and deep learning (DL) can be used to track and analyze the underlying image texture and features to predict the Hoehn-Yahr stages (HYS) of PD. In this study, we aim to predict HYS at year 0 and year 4 after the first diagnosis with combined imaging, radiomics and DL-based features using 123I-Ioflupane SPECT images at year 0. Methods: In this study, 161 subjects from the Parkinson's Progressive Marker Initiative database underwent baseline 3T MRI and 123I-Ioflupane SPECT, with HYS assessment at years 0 and 4 after first diagnosis. Conventional imaging features (IF) and radiomic features (RaF) for striatum uptakes were extracted from SPECT images using MRI- and SPECT-based (SPECT-V and SPECT-T) segmentations respectively. A 2D DenseNet was used to predict HYS of PD, and simultaneously generate deep features (DF). The random forest algorithm was applied to develop models based on DF, RaF, IF and combined features to predict HYS (stage 0, 1 and 2) at year 0 and (stage 0, 1 and ≥ 2) at year 4, respectively. Model predictive accuracy and receiver operating characteristic (ROC) analysis were assessed for various prediction models. Results: For the diagnostic accuracy at year 0, DL (0.696) outperformed most models, except DF + IF in SPECT-V (0.704), significantly superior based on paired t-test. For year 4, accuracy of DF + RaF model in MRI-based method is the highest (0.835), significantly better than DF + IF, IF + RaF, RaF and IF models. And DL (0.820) surpassed models in both SPECT-based methods. The area under the ROC curve (AUC) highlighted DF + RaF model (0.854) in MRI-based method at year 0 and DF + RaF model (0.869) in SPECT-T method at year 4, outperforming DL models, respectively. And then, there was no significant differences between SPECT-based and MRI-based segmentation methods except for the imaging feature models. Conclusion: The combination of radiomic and deep features enhances the prediction accuracy of PD HYS compared to only radiomics or DL. This suggests the potential for further advancements in predictive model performance for PD HYS at year 0 and year 4 after first diagnosis using 123I-Ioflupane SPECT images at year 0, thereby facilitating early diagnosis and treatment for PD patients. No significant difference was observed in radiomics results obtained between MRI- and SPECT-based striatum segmentations for radiomic and deep features. [ABSTRACT FROM AUTHOR]

The classification of medical images is crucial in the biomedical field, and despite attempts to address the issue, significant challenges persist. To effectively categorize medical images, collecting and integrating statistical information that accurately describes the image is essential. This study proposes a uniquemethod for feature extraction that combines deep spatial characteristics with handmade statistical features. The approach involves extracting statistical radiomics features using advanced techniques, followed by a novel handcrafted feature fusion method inspired by the ResNet deep learning model. A new feature fusion framework (FusionNet) is then used to reduce image dimensionality and simplify computation. The proposed approach is tested on MRI images of brain tumors fromthe BraTS dataset, and the results showthat it outperforms existingmethods regarding classification accuracy. The study presents three models, including a handcrafted-based model and two CNN models, which completed the binary classification task. The recommended hybrid approach achieved a high F1 score of 96.12 ± 0.41, precision of 97.77 ± 0.32, and accuracy of 97.53 ± 0.24, indicating that it has the potential to serve as a valuable tool for pathologists. [ABSTRACT FROM AUTHOR]

Correlation filter tracking algorithms have garnered significant attention due to their efficiency and outstanding tracking performance. However, these methods face several limitations. Firstly, they rely on periodic boundary conditions, leading to boundary effects. Secondly, most traditional methods only extract hand-crafted features from the image. Nevertheless, these features are insufficient to discriminate in complex scenes. Thirdly, they assume that the maximum position of the correlation response represents the object without further evaluating its reliability. These limitations make it very easy to lose the object in occluded scenes. A background-aware correlation filter algorithm based on an anti-occlusion mechanism and deep features is proposed to solve the above limitations. Primarily, the video frames to be processed are cyclically shifted to crop the cyclically shifted samples in a small window. This operation significantly reduces boundary effects while obtaining background samples from the real world. Then, deep features are extracted through deep neural networks. Subsequently, we propose a background perception tracking framework that synchronously estimates position and scale based on these features. This framework aims to determine the optimal candidate sample position and scale. Finally, an anti-occlusion mechanism is constructed to evaluate the optimal candidate samples obtained in each frame further. This mechanism fully exploits the diversity of objects and effectively solves the tracking drift and failure issues caused by occlusion, fast motion, and so on. Extensive experiments are conducted on the object tracking benchmark (OTB) dataset and compared with industry-leading tracking algorithms to validate the effectiveness of the proposed method. The results show that the method has robust tracking performance. [ABSTRACT FROM AUTHOR]

Accurate detection of defects on steel surfaces is crucial for maintaining quality standards in steel production. This paper addresses the challenge of classifying steel sheets into distinct defect categories by presenting a robust method that leverages deep learning and advanced optimization techniques. We propose a novel approach that utilizes the ResNet101 model to extract deep features, which are then classified using a support vector machine (SVM). To enhance the SVM's performance, Bayesian optimization is employed for hyperparameter tuning. Our method is validated using the ''Severstal: Steel Defect Detection'' dataset from Kaggle, achieving a validation accuracy of 89.1 % and a test accuracy of 90.6 %, with a classification error of 0.10934. Additionally, the area under the curve (AUC) for each class exceeds 0.95 in both the validation and test sets, demonstrating excellent discriminatory power. Further evaluation on the DAGM dataset achieved flawless results, with an accuracy of 100 %, AUC of 1, sensitivity of 100 %, specificity of 100 %, precision of 100 %, MCC of 100 %, F1 score of 1, and kappa of 100 %. On the NEU dataset, our method achieved an accuracy of 97.92 %, sensitivity of 97.92 %, specificity of 99.58 %, precision of 98.06 %, F1 score of 0.9791, MCC of 97.55 %, and kappa of 92.50 %. These results demonstrate the robustness and adaptability of the proposed method, offering an efficient and reliable solution for automating steel defect detection and surface defect classification in industrial applications.

In the context of addressing new image classification tasks with insufficient training samples via pre-trained deep learning models, the methods based on the Bag-of-Deep-Visual-Words (BoDVW) model have achieved higher classification accuracy across various image classification tasks compared to directly using the new classification layer of the pre-trained model for classification. These methods perform a sequence of operations on the input image - deep feature extraction, feature encoding, and feature pooling - to obtain an image representation vector, which is then fed into classifiers for classification. However, they ignore two crucial aspects: the high-level semantic characteristics of deep features and their local context within the feature space, which limits the image classification performance. To address this issue, we propose a new image classification method with a unique workflow. Specifically, our method identifies low-entropy local regions in the feature space by constructing multiple decision trees, using the set of labelled deep features built from training images. For a given image, the voting vector of each deep feature from the image is calculated based on the category label distributions of the low-entropy local regions where it is located. This vector reflects the degree of support that the feature provides for the hypothesis that it belongs to each category. The voting vectors of all features are aggregated according to image regions of different sizes and positions to obtain the representation vector of the image. The representation vectors of testing images are input into Support Vector Machines (SVMs) trained using those of training images to predict their categories. Experimental results on six public datasets show that our method achieves higher classification accuracy by 0.07% to 3.6% (averaging at 0.8%) compared to two BoDVW methods, and by 0.1% to 10.69% (averaging at 2.69%) compared to directly using the new classification layer of the pre-trained model for classification. These results demonstrate the effectiveness of considering the high-level semantic characteristics of deep features and their local context within the feature space for image classification. Importantly, the unique workflow of our method opens up new potential avenues for improving classification performance. These include increasing the number of local regions where deep features primarily originate from one or a few image categories, improving the accuracy of low-entropy local region identification, and developing an end-to-end deep learning model based on this workflow. While maintaining classification accuracy comparable to recent works, our method offers notable potential for the advancement of the image classification field. [ABSTRACT FROM AUTHOR]

Objective To construct a precise model for identifying traditional Chinese medicine (TCM) constitutions, thereby offering optimized guidance for clinical diagnosis and treatment planning, and ultimately enhancing medical efficiency and treatment outcomes. Methods First, TCM full-body inspection data acquisition equipment was employed to collect full-body standing images of healthy people, from which the constitutions were labelled and defined in accordance with the Constitution in Chinese Medicine Questionnaire (CCMQ), and a dataset encompassing labelled constitutions was constructed. Second, heat-suppression valve (HSV) color space and improved local binary patterns (LBP) algorithm were leveraged for the extraction of features such as facial complexion and body shape. In addition, a dual-branch deep network was employed to collect deep features from the full-body standing images. Last, the random forest (RF) algorithm was utilized to learn the extracted multifeatures, which were subsequently employed to establish a TCM constitution identification model. Accuracy, precision, and F1 score were the three measures selected to assess the performance of the model. Results It was found that the accuracy, precision, and F1 score of the proposed model based on multifeatures for identifying TCM constitutions were 0.842, 0.868, and 0.790, respectively. In comparison with the identification models that encompass a single feature, either a single facial complexion feature, a body shape feature, or deep features, the accuracy of the model that incorporating all the aforementioned features was elevated by 0.105, 0.105, and 0.079, the precision increased by 0.164, 0.164, and 0.211, and the F1 score rose by 0.071, 0.071, and 0.084, respectively. Conclusion The research findings affirmed the viability of the proposed model, which incorporated multifeatures, including the facial complexion feature, the body shape feature, and the deep feature. In addition, by employing the proposed model, the objectification and intelligence of identifying constitutions in TCM practices could be optimized. [ABSTRACT FROM AUTHOR]

In hyperspectral image (HSI) reconstruction tasks, due to the lack of ground truth in real imaging processes, models are usually trained and validated on simulation datasets and then tested on real measurements captured by real HSI imaging systems. However, due to the gap between the simulation imaging process and the real imaging process, the best model validated on the simulation dataset may fail on real measurements. To obtain the best model for the real-world task, it is crucial to design a suitable no-reference HSI quality assessment metric to reflect the reconstruction performance of different models. In this paper, we propose a novel no-reference HSI quality assessment metric via ranking feature learning (R-NHSIQA), which calculates the Wasserstein distance between the distribution of the deep features of the reconstructed HSIs and the benchmark distribution. Additionally, by introducing the spectral self-attention mechanism, we propose a Spectral Transformer (S-Transformer) to extract the spatial-spectral representative deep features of HSIs. Furthermore, to extract quality-sensitive deep features, we use quality ranking as a pre-training task to enhance the representation capability of the S-Transformer. Finally, we introduce the Wasserstein distance to measure the distance between the distribution of the deep features and the benchmark distribution, improving the assessment capacity of our method, even with non-overlapping distributions. The experimental results demonstrate that the proposed metric yields consistent results with multiple full-reference image quality assessment (FR-IQA) metrics, validating the idea that the proposed metric can serve as a substitute for FR-IQA metrics in real-world tasks. [ABSTRACT FROM AUTHOR]

Food classification has always piqued the interest of both domestic and international researchers, but numerous challenges remain. We present the dataset UIT-TASTET21, which contains over 77,000 color images of 18 traditional Vietnamese Lunar New Year dishes. We have experimented with classification using feature vectors from network architectures such as VGG16, Inception-v3, ResNet-50, Xception, and MobileNet-v2 to train support vector machines (SVMs), meeting the dataset’s challenges and laying the groundwork for the development of many optimal methods in the future that promise scientific breakthroughs in the service and commercial industries. At the same time, the authors desire to share a piece of Vietnamese cuisine’s distinctiveness with worldwide friends.

Simple Summary: Animal movement trajectories are effective indicators of key information such as social behavior, food acquisition, reproduction, migration, and survival strategies in animal behavior analysis. However, manual observation is still relied upon in many analysis scenarios, which is inefficient and error-prone. This paper introduces a computer vision-based method for tracking animal trajectories, which enables monitoring and accurate acquisition of individual target animal movement trajectories over extended periods, overcoming the limitations of manual observation. The experiments demonstrate that the method is efficient and accurate in tracking animals in complex scenes, providing essential basic data for animal behavior analysis and having a wide range of potential applications. Animal tracking is crucial for understanding migration, habitat selection, and behavior patterns. However, challenges in video data acquisition and the unpredictability of animal movements have hindered progress in this field. To address these challenges, we present a novel animal tracking method based on correlation filters. Our approach integrates hand-crafted features, deep features, and temporal context information to learn a rich feature representation of the target animal, enabling effective monitoring and updating of its state. Specifically, we extract hand-crafted histogram of oriented gradient features and deep features from different layers of the animal, creating tailored fusion features that encapsulate both appearance and motion characteristics. By analyzing the response map, we select optimal fusion features based on the oscillation degree. When the target animal's state changes significantly, we adaptively update the target model using temporal context information and robust feature data from the current frame. This updated model is then used for re-tracking, leading to improved results compared to recent mainstream algorithms, as demonstrated in extensive experiments conducted on our self-constructed animal datasets. By addressing specific challenges in animal tracking, our method offers a promising approach for more effective and accurate animal behavior research. [ABSTRACT FROM AUTHOR]

A CME-based few-shot detection model with enhanced multiscale deep feature is proposed to address the problems that existing few-shot detection models have insufficient consideration of global semantic information of images and degradation of detector performance due to varying input image sizes. Firstly, the model is trained with a large amount of labeled base class data and a multilayer convolutional neural network based on residual jumping and a multiscale feature enhanced module with good generalization, then the model is fine- tuned with a small amount of labeled new class data and base class data, and finally the fine-tuned model is used for target detection. To verify the effectiveness of the model, the VOC2007 and VOC2012 datasets are used to train and evaluate the model, and the relevant ablation experiments demonstrate that the introduction of a multilayer convolutional neural network with residual jump structure and the multiscale feature enhanced module can further increase the accuracy of the model both alone and in combination. In comparison experiments with six representative small-sample target detection models, it is shown that the CME with multiscale deep feature deepening scores outperforms the state-of-the-art detector by an average of 4.75 percentage points. [ABSTRACT FROM AUTHOR]

Mineral classification is a crucial task for geologists. Minerals are identified by their characteristics. In the field, geologists can identify minerals by examining lustre, color, streak, hardness, crystal habit, cleavage, fracture, and specific features. Geologists sometimes use a magnifying hand lens to identify minerals in the field. Surface color can assist in identifying minerals. However, it varies widely, even within a single mineral family. Some minerals predominantly show a single color. So, identifying minerals is possible considering surface color and texture. But, again, a limited database of minerals is available with large-scale images. So, the challenges arise to identify the minerals using their images with limited images. With the advancement of machine learning, the deep learning approach with bi-layer feature fusion enhances the dimension of the feature vector with the possibility of high accuracy. Here, an experimental analysis is reported with three possibilities of bi-layer feature fusion of three CNN models like Alexnet, VGG16 & VGG19, and a framework is suggested. Alexnet delivers the highest performance with the bi-layer fusion of fc6 and fc7. The achieved accuracy is 84.23%, sensitivity 84.23%, specificity 97.37%, precision 84.7%, FPR 2.63%, F1 Score 84.17%, MCC 81.75%, and Kappa 53.59%. [ABSTRACT FROM AUTHOR]

Recent studies have shown that deep neural networks (DNNs) are vulnerable to adversarial examples (AEs). Denoising based on the input pre-processing is one of the defenses against adversarial attacks. However, it is hard to remove multiple adversarial perturbations, especially in the presence of evolving attacks. To address this challenge, we attempt to extract the commonality of adversarial perturbations. Due to the imperceptibility of adversarial perturbations in the input space, we conduct the extraction in the deep feature space where the perturbations become more apparent. Through the obtained common characteristics, we craft common adversarial examples (CAEs) to train the denoiser. Furthermore, to prevent image distortion while removing as much of the adversarial perturbation as possible, we propose a hybrid loss function that guides the training process at both the pixel level and the deep feature space. Our experiments show that our defense method can eliminate multiple adversarial perturbations, significantly enhancing adversarial robustness compared to previous state-of-the-art methods. Moreover, it can be plug-and-play for various classification models, which demonstrates the generalizability of our defense method. [ABSTRACT FROM AUTHOR]

Introduction: This study introduces a novel methodology for classifying human papillomavirus (HPV) using colposcopy images, focusing on its potential in diagnosing cervical cancer, the second most prevalent malignancy among women globally. Addressing a crucial gap in the literature, this study highlights the unexplored territory of HPV-based colposcopy image diagnosis for cervical cancer. Emphasising the suitability of colposcopy screening in underdeveloped and low-income regions owing to its small, cost-effective setup that eliminates the need for biopsy specimens, the methodological framework includes robust dataset augmentation and feature extraction using Efficient- NetB0 architecture. Material and methods: The optimal convolutional neural network model was selected through experimentation with 19 architectures, and fine-tuning with the fine κ-nearest neighbour algorithm enhanced the classification precision, enabling detailed distinctions with a single neighbour. Results: The proposed methodology achieved outstanding results, with a validation accuracy of 99.9% and an area under the curve (AUC) of 99.86%, with robust performance on test data, 91.4% accuracy, and an AUC of 91.76%. These remarkable findings underscore the effectiveness of the integrated approach, which offers a highly accurate and reliable system for HPV classifi- cation. Conclusions: This research sets the stage for advancements in medical imaging applications, prompting future refinement and validation in diverse clinical settings. [ABSTRACT FROM AUTHOR]

Fire poses a significant threat to both lives and property, necessitating effective early detection measures. Despite challenges in identifying smoke and fire in their initial stages, we have devised a cost-efficient visual detection system. Early fire detection enhances its potential effectiveness. CCTV surveillance systems are now commonplace in developed countries, serving as tools for periodic monitoring of various locations. However, fluctuating ambient light conditions, camera angles, and seasonal variations can introduce data distortions, occlusions, and impact model accuracy. To address these issues, we've implemented a method combining deep learning networks and machine learning strategies for flame detection and direction classification. Our innovative QuickDenseNet extracts dense features from segmented flame video frames. We introduce the Ensemble Score Voted SVM (ESV-SVM), employing SVM as the primary learner and score voting as the auxiliary learner. Our approach is rigorously evaluated through simulations, measuring accuracy and various Key Performance Indices (KPIs), including Precision, F1-score, Recall, Correlation, Error, FPR, and Correlation Coefficients. Remarkably, our proposed method achieves an impressive precision rate of approximately 99.5%. [ABSTRACT FROM AUTHOR]

The classification of medical images is crucial in the biomedical field, and despite attempts to address the issue, significant challenges persist. To effectively categorize medical images, collecting and integrating statistical information that accurately describes the image is essential. This study proposes a unique method for feature extraction that combines deep spatial characteristics with handmade statistical features. The approach involves extracting statistical radiomics features using advanced techniques, followed by a novel handcrafted feature fusion method inspired by the ResNet deep learning model. A new feature fusion framework (FusionNet) is then used to reduce image dimensionality and simplify computation. The proposed approach is tested on MRI images of brain tumors from the BraTS dataset, and the results show that it outperforms existing methods regarding classification accuracy. The study presents three models, including a handcrafted-based model and two CNN models, which completed the binary classification task. The recommended hybrid approach achieved a high F1 score of 96.12 ± 0.41, precision of 97.77 ± 0.32, and accuracy of 97.53 ± 0.24, indicating that it has the potential to serve as a valuable tool for pathologists.

Deblurring is a classical image restoration problem. Although recent methods have shown promising deblurring performance, most methods still cannot effectively balance the texture details restoration and model complexity. In order to improve the performance of deblurring, some models are designed to be more complex. In this work, a simple and efficient Wiener deconvolution and multi-scale transformer-based U-Net (WTransU-Net) is proposed to tackle these problems. First, the proposed Wiener feature extraction module uses explicit Wiener deconvolution to extract the Wiener features in the deep feature space. Then, the obtained Wiener features are input into a multi-scale feature reconstruction module which only embeds one transformer refining block in each scale of the U-Net to deblur the image from local and global perspectives. In addition, a multi-scale hybrid loss function is designed to train the WTransU-Net in an end-to-end manner to better learn the content and texture details. The experimental results on benchmark datasets show that compared with the state-of-the-art deblurring methods, the proposed WTransU-Net can achieve better performance with fewer artifacts in terms of quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

In hyperspectral image (HSI) reconstruction tasks, due to the lack of ground truth in real imaging processes, models are usually trained and validated on simulation datasets and then tested on real measurements captured by real HSI imaging systems. However, due to the gap between the simulation imaging process and the real imaging process, the best model validated on the simulation dataset may fail on real measurements. To obtain the best model for the real-world task, it is crucial to design a suitable no-reference HSI quality assessment metric to reflect the reconstruction performance of different models. In this paper, we propose a novel no-reference HSI quality assessment metric via ranking feature learning (R-NHSIQA), which calculates the Wasserstein distance between the distribution of the deep features of the reconstructed HSIs and the benchmark distribution. Additionally, by introducing the spectral self-attention mechanism, we propose a Spectral Transformer (S-Transformer) to extract the spatial-spectral representative deep features of HSIs. Furthermore, to extract quality-sensitive deep features, we use quality ranking as a pre-training task to enhance the representation capability of the S-Transformer. Finally, we introduce the Wasserstein distance to measure the distance between the distribution of the deep features and the benchmark distribution, improving the assessment capacity of our method, even with non-overlapping distributions. The experimental results demonstrate that the proposed metric yields consistent results with multiple full-reference image quality assessment (FR-IQA) metrics, validating the idea that the proposed metric can serve as a substitute for FR-IQA metrics in real-world tasks.

Writer identification based on handwriting recognition is considered one of the most common research areas in pattern recognition and biometrics. It has attracted much attention in recent decades due to the urgent need to develop biometric systems for many security applications. In this paper, Deep Writer Identification Network (DeepWINet), an effective deep Convolutional Neural Network (CNN) for writer identification, is proposed. The proposed model is evaluated in two different ways. In the first scenario, DeepWINet's CNN activation features, computed from the connected components of the writing, are passed to a customized nearest neighbor classifier for writer identification. In the second scenario, DeepWINet is evaluated as an end-to-end CNN network where the predicted results are averaged using an efficient strategy, Score Averaging Component-Decision Combiner. The proposed approach achieves competitive or the highest State-Of-The-Art performance on eight challenging handwritten databases with different languages. [ABSTRACT FROM AUTHOR]

Tuna resources are an important part of China's pelagic fishery production. However, for China's tuna fishery, tuna species caught at sea are still manually classified, which is a time-consuming and inefficient process; so China's tuna fishery needs to develop toward automation. This study uses gray-level co-occurrence matrix (GLCM) and VGG16 to visualize phenotypic texture through local images of three Thunnus species. At the same time, texture feature index data (TFD), deep feature data (DFD), and their combined feature data (CFD) are obtained from texture images. Support vector machine (SVM) with different kernel functions is used to classify phenotypic texture of tuna automatically. The study shows that visualized texture images of different tuna using GLCM and VGG16 have biological characteristics. In the classification results without cross-validation, the average classification accuracy of TFD in polynomial was 83%, the average classification accuracy of DFD in RBF (Radial basis function) was 93%, and the average classification accuracy of CFD in RBF was 95%. It is concluded that tuna phenotype texture can be efficiently classified by using SVM with different kernel functions. [ABSTRACT FROM AUTHOR]

Robust feature matching for multimodal remote-sensing images remains challenging due to the significant nonlinear radiation difference (NRD) caused by modality variations. In this letter, we present a novel feature-matching method for multimodal remote-sensing images, called robust deep feature matching (RDFM), which exploits only deep features extracted by a pretrained Visual Geometry Group (VGG) network to achieve competitive performance. It is shown that template matching of these pretrained features is robust to NRD for various multimodal remote-sensing images, and no additional training is required to improve the matching performance. To extract as many correspondences as possible, we use dense template matching to obtain point correspondences and introduce a 4-D convolution-based implementation of dense template matching for the sake of computational efficiency. RDFM consists of two main steps. First, enormous coarse correspondences are extracted by applying dense template matching at the deep layer of the pretrained network, and then a coarse-to-fine hierarchical refinement is performed to obtain high-quality correspondences. To verify the effectiveness of RDFM, six different types of multimodal image datasets are used in our experiments, including day–night, depth–optical, infrared–optical, map–optical, optical–optical, and SAR–optical datasets. The comprehensive experimental results show that RDFM can overcome the problem caused by NRD and achieves a better performance than the state-of-the-art methods for multimodal remote-sensing image matching. The code of RDFM is publicly available at https://github.com/Fans2017/RDFM. [ABSTRACT FROM AUTHOR]

Recent advances in deep learning have enabled state-of-the-art performance in detecting medium and large-size objects. However, small object detection remains challenging primarily due to the scarcity of information. This paper proposes an end-to-end fusion network that integrates deep and hand-crafted features to address this limitation. A fusion module based on semantic context information is designed to enhance feature discrimination ability. Additionally, we introduce a kind of feature-contrast loss to incorporate prior knowledge into the learning of deep feature according to contrastive learning. Experiments on MS COCO (34.4% ${\mathrm {A}}{{\mathrm {P}}_{\mathrm {S}}}$ ) and PASCAL VOC (85.9% mAP) datasets demonstrate that our approach achieves improved detection accuracy over previous methods, especially for small objects.

Retargeting aims at displaying a photo with an arbitrary aspect ratio, wherein the visually/semantically prominent objects are appropriately preserved and visual distortions can be well alleviated. Conventional retargeting models are built upon the visual perception of photos from a family of prespecified communities (e.g., “portrait”), wherein the underlying community-specific features are not learned explicitly. Thus, they cannot appropriately retarget aerial photos, which contains a rich variety of objects with different scales. In this article, a novel aerial photo retargeting framework is designed by encoding the deep features from automatically detected Google Maps (https://www.google.com/maps) communities into a regularized probabilistic model. Specifically, we first propose an enhanced matrix factorization (MF) algorithm to calculate communities based on million-scale Google Maps pictures, for each of which deep feature is learned simultaneously. The enhanced MF incorporates label denoising, between-communities correlation, and deep feature encoding collaboratively. Subsequently, a probabilistic model called latent topic model (LTM) is designed that quantifies the spatial layouts of multiple Google Maps communities in the underlying hidden space. To alleviate the overfitting from Google Maps communities with imbalanced numbers of aerial photos, a regularizer is added into the LTM. Finally, by leveraging the regularized LTM, we shrink the test photo horizontally/vertically to maximize the posterior probability of the retargted photo. Comprehensive subjective evaluations and visualizations have demonstrated the advantages of our method. Besides, our calculate Google Maps communities are competitively consistent with the ground truth, according to the quantitative comparisons on the 2 M Google Maps photos.

Animal tracking is crucial for understanding migration, habitat selection, and behavior patterns. However, challenges in video data acquisition and the unpredictability of animal movements have hindered progress in this field. To address these challenges, we present a novel animal tracking method based on correlation filters. Our approach integrates hand-crafted features, deep features, and temporal context information to learn a rich feature representation of the target animal, enabling effective monitoring and updating of its state. Specifically, we extract hand-crafted histogram of oriented gradient features and deep features from different layers of the animal, creating tailored fusion features that encapsulate both appearance and motion characteristics. By analyzing the response map, we select optimal fusion features based on the oscillation degree. When the target animal’s state changes significantly, we adaptively update the target model using temporal context information and robust feature data from the current frame. This updated model is then used for re-tracking, leading to improved results compared to recent mainstream algorithms, as demonstrated in extensive experiments conducted on our self-constructed animal datasets. By addressing specific challenges in animal tracking, our method offers a promising approach for more effective and accurate animal behavior research.

Multimodal emotion recognition aims to recognize human emotional states through different modalities related to human emotion expression such as audio, vision, text, etc. This topic is of great importance in the fields of human-computer interaction, a.pngicial intelligence, affective computing, etc., and has attracted much attention. In view of the great success of deep learning methods developed in recent years in various tasks, a variety of deep neural networks have been used to learn high-level emotional feature representations for multimodal emotion recog-nition. In order to systematically summarize the research advance of deep learning methods in the field of multi-modal emotion recognition, this paper aims to present comprehensive analysis and summarization on recent multi-modal emotion recognition literatures based on deep learning. First, the general framework of multimodal emotion recognition is given, and the commonly used multimodal emotional dataset is introduced. Then, the principle of representative deep learning techniques and its advance in recent years are briefly reviewed. Subsequently, this paper focuses on the advance of two key steps in multimodal emotion recognition: emotional feature extraction methods related to audio, vision, text, etc., including hand-crafted feature extraction and deep feature extraction; multi-modal information fusion strategies integrating different modalities. Finally, the challenges and opportunities in this field are analyzed, and the future development direction is pointed out.

Tool wear leads to a reduction in dimensional accuracy and surface quality, as well as unexpected sudden tool failure. A broken tool can cause irreparable damage to an expensive workpiece, resulting in increased operating costs and production delays. Since the mechanical strength of small-diameter drills is inadequate for the load and prone to breakage, tool condition monitoring and diagnosis is important to prevent sudden tool breakage, increase productivity, and promote automation in machining process. The present work is aimed to investigate a tool condition monitoring method based on the analysis of acoustic emission (AE) signals emitted during small-hole drilling. We propose DDM (Deep feature Distribution Modeling), a method for image-level anomaly detection and anomaly segmentation in time-series signal analysis. The peck drilling experiments on SKD61 steels were performed with high-speed steel (HSS) drills. The continuous wavelet transform (CWT) was applied to generate time-frequency (TF) image of the AE signals during the drilling process. The TF images were quantified as anomaly scores using the DDM, which establishes normality by fitting a multivariate Gaussian (MVG) to pre-trained deep features. The anomaly detection capability of the DDM and the convolutional autoencoder (CAE) was compared using dummy data for validation. The digital microscope was employed to measure tool wear. Chip morphology was also observed by the laser microscopy. As the tool wear progressed, the anomaly score increased or decreased, with several sharp increases observed between holes 3805 and 3869 just prior to tool failure. An increase in the width of the shear layer spacing of the chips was also observed just prior to failure. Changes in the anomaly score associated with tool wear were more clearly identified by creating anomaly maps. The present investigation shows that waveform processing of AE signals using the CWT and anomaly detection based on the DDM are efficient methods for tool condition monitoring. Our proposed approach makes it possible to visualize the differences in anomaly states using a more subdivided layer context by generating multiple anomaly maps with deep feature vectors obtained from multiple layers.

Applying deep convolutional features to image retrieval has become the mainstream method in the field of image retrieval. However, the discriminative power of deep convolutional features needs to be further improved. So far, low-level features have not been used to enhance representation. Furthermore, the compatibility of low-level features and deep features remains challenging. To address these problems, we propose an aggregation method that combines low-level features and deep features for image retrieval. The highlights are: (1) We propose a novel similarity weight to complement the advantages of the intrinsic information in each feature type, thereby improving the discriminative power of deep convolutional features. (2) We introduce a feature loss weight to calculate the diversity between texture features and deep features, which effectively utilizes the complementary advantages of each feature type. (3) We propose a novel representation for image retrieval, named the dual-weighted deep feature descriptor. It not only strengthens the discriminative power of the intrinsic information within feature maps, but also avoids the spatial weighting from mistakenly removing useful information. Experiments demonstrate that our method outperforms some state-of-the-art methods on five benchmark datasets. It is not only simpler to implement, as it does not require complex processes such as retraining a convolutional neural network model, but also has low experimental requirements. [ABSTRACT FROM AUTHOR]

The papaya (Carica papaya L.) is a tropical fruit with high commercial value due to its superior nutritional and therapeutic properties. Papayas must be packaged in the fruit industry according to their degree of ripeness. Physically grading papaya fruit using human vision is time-consuming and destructive. A brand-new, non-destructive classification system for papaya fruit development stages is being offered as a result of this study. The project proposes to investigate three classification models: one deep learning method, the DWT approach, and a hybrid approach. A total of 300 papaya fruit sample photos were used in the experiment, 100 of which corresponded to each fruit's three ripeness stages: midripen, ripen, and un-ripen. The maturity level of papaya is estimated using a hybrid network, i.e., the combination of high-level features and an SVM classifier. The high-level features are the integration of deep Features of VGG16 and coefficients of DWT. The accuracy and AUC of the proposed hybrid model are 98% and 100%, respectively. [ABSTRACT FROM AUTHOR]

Deep features and texture features each have advantages in image representation. However, exploiting deep textures for image retrieval is challenging because it is difficult to enhance the compatibility of texture features and deep features. To address this problem, we propose a novel image-retrieval method named the deep texture feature histogram (DTFH). The main highlights are: (1) We propose a novel method for identifying effective, limited-effectiveness, or non-valid feature maps via ranking based on Haralick's statistics, which can help understand image content and identify objects, as these statistics have clear physical significance. (2) We use Gabor filtering to mimic the human orientation-selection mechanism, which allows deep texture features to contain a good representation of orientation, thereby enhancing discriminative power. (3) We combine the advantages of classical texture features and deep features to provide a compact representation. This provides a new, yet simple way to exploit deep features via the use of traditional classical texture features. Comparative experiments demonstrate that deep texture features provide highly competitive performance in image retrieval in terms of mean average precision (mAP), and provide new insights into the exploitation of traditional texture features and deep features. [ABSTRACT FROM AUTHOR]

Aiming at the problems of high algorithm time complexity and incomplete location region in image copy-move forgery detection, an image copy-move forgery detection algorithm based on deep features extraction and discrete cosine transform is proposed. Firstly, the color and texture information of the image are fused to obtain the four-channel image, the adaptive feature extraction threshold is calculated, and the depth feature is extracted by the feature detector based on the full convolutional neural network. Secondly, the discrete cosine transform is used to extract block features for preliminary matching, and point feature vectors are adopted to eliminate mismatches. Finally, the tampered regions are accurately located by convolution operation. The verification on the public dataset fully demonstrates the advantages of the algorithm in the detection efficiency and the integrity of the positioning area. [ABSTRACT FROM AUTHOR]

The anomaly detection of aeroengines faces significant challenges, including high noise, complex parameter correlations, and imbalanced data. Current methods primarily rely on the reconstruction error of autoencoders to isolate anomalies. However, such methods are ineffective in detecting minor anomalies, as the reconstruction error for minor anomaly samples closely resembles that of normal samples. To address this problem, this paper proposes an innovative Deep Autoencoder Anomaly Detection Model (DAADM) for the anomaly detection of aeroengines. DAADM consists of two sub-networks: Anomaly Score Calculation Network (ASCN) and Deep Feature Extraction Network (DFEN). Firstly, ASCN introduces the adversarial training strategy into deep autoencoders, ensuring training stability while effectively isolating minor anomalies. Secondly, DFEN extracts deep-level features from input samples, providing a different perspective from ASCN to enhance sample information extraction. The combination of ASCN and DFEN compensates for their respective deficiencies, comprehensively considering input sample information, thereby promoting the capability of anomaly detection. Finally, the proposed DAADM is validated on real an aeroengine dataset and a public dataset. On the aeroengine dataset, the accuracy, recall, and F1 score of DAADM exceed the state-of-the-art method by 4.73%, 4.79%, and 5.76%, respectively. It is noteworthy that experimental results have proven that DAADM has strong anti-noise capabilities. [ABSTRACT FROM AUTHOR]