Search

The 'black box' characteristics of Convolutional Neural Networks (CNNs) present significant risks to their application scenarios, such as reliability, security, and division of responsibilities. Addressing the interpretability of CNN emerges as an urgent and critical issue in the field of machine learning. Recent research on CNN interpretability has either yielded unstable or inconsistent interpretations, or produced coarse-scale interpretable heatmaps, limiting their applicability in various scenarios. In this work, we propose a novel method of CNNs interpretation by incorporating a joint evaluation of multiple feature maps and employing multi-objective optimization (JE&MOO-CAM). Firstly, a method of joint evaluation for all feature maps is proposed to preserve the complete object instances and improve the overall activation values. Secondly, an interpretation method of CNNs under the MOO framework is proposed to avoid the instability and inconsistency of interpretation. Finally, the operators of selection, crossover, and mutation, along with the method of population initialization in NSGA-II, are redesigned to properly express the characteristics of CNNs. The experimental results, including both qualitative and quantitative assessments along with a sanity check conducted on three classic CNN models—VGG16, AlexNet, and ResNet50—demonstrate the superior performance of the proposed JE&MOO-CAM model. This model not only accurately pinpoints the instances within the image requiring explanation but also preserves the integrity of these instances to the greatest extent possible. These capabilities signify that JE&MOO-CAM surpasses six other leading state-of-the-art methods across four established evaluation criteria. [ABSTRACT FROM AUTHOR]

Quantum machine learning, an emerging field at the intersection of quantum computing and classical machine learning, has shown great promise in enhancing computational capabilities beyond classical bounds. A key element in this area of research is the utilization of Quantum Kernel Estimators, traditionally grounded in the symmetries of SU(2) groups associated with qubits. This study extends the conceptual framework of Quantum Kernel Estimators to incorporate a broader spectrum of symmetry groups. By harnessing the various structures of Lie groups, we develop novel quantum-inspired feature maps that offer more flexible and potentially powerful ways to encode and compress classical data into quantum states. We present a comprehensive theoretical introduction for this approach, followed by a methodology that integrates the developed feature maps into quantum-inspired kernel classifiers. Our results, derived from a series of computational experiments across various datasets, demonstrate the efficacy of this approach in comparison to traditional quantum and classical machine learning models. The findings not only underline the versatility of Lie-group theory in potentially enhancing quantum machine learning algorithms but also open new avenues for exploring complex symmetries in quantum information processing. This research bridges a gap between the study of symmetries and machine learning, paving the way for more sophisticated quantum algorithms capable of tackling complex, high-dimensional data in ways previously unattainable. • We present a construction of a novel data encoding technique based on quantum feature maps using a variety of Lie groups. • This approach is shown to work effectively on four publicly accessible datasets and on synthetic data set. • We provide the full code repository to construct feature maps on arbitrary Lie groups and kernel matrices. • We discuss possible applications and limitations of our ideas. [ABSTRACT FROM AUTHOR]

The enormous data and computational resources required by Convolutional Neural Networks (CNNs) hinder the practical application on mobile devices. To solve this restrictive problem, filter pruning has become one of the practical approaches. At present, most existing pruning methods are currently developed and practiced with respect to the spatial domain, which ignores the potential interconnections in the model structure and the decentralized distribution of image energy in the spatial domain. The image frequency domain transform method can remove the correlation between image pixels and concentrate the image energy distribution, which results in lossy compression of images. In this study, we find that the frequency domain transform method is also applicable to the feature maps of CNNs. The filter pruning via wavelet transform (WT) is proposed in this paper (FPWT), which combines the frequency domain information of WT with the output feature map to more obviously find the correlation between feature maps and make the energy into a relatively concentrated distribution in the frequency domain. Moreover, the importance score of each feature map is calculated by the cosine similarity and the energy-weighted coefficients of the high and low frequency components, and prune the filter based on its importance score. Experiments on two image classification datasets validate the effectiveness of FPWT. For ResNet-110 on CIFAR-10, FPWT reduces FLOPs and parameters by more than 60.0 % with 0.53 % accuracy improvement. For ResNet-50 on ImageNet, FPWT reduces FLOPs by 53.8 % and removes parameters by 49.7 % with only 0.97 % loss of Top-1 accuracy. [ABSTRACT FROM AUTHOR]

• The time-domain pulse wave is transformed into a feature map of 36-dimensional frequency-domain mel-frequency cepstral coefficients (MFCC), and a pre-training network based on small-sample specific targets is constructed to enhance feature learning capability of pulse wave. • The fusion attention mechanism is added to the inverted residual to improve the global feature correlation. 3 × 3 convolutional and BN layers are added to reduce overfitting. • We study the correlations between the temporal and frequency domain characteristics of pulse wave and hypertensive classification of hypertensive target organ damage(TOD), and analyze the key features of pulse wave affecting TOD in hypertension, so as to provide effective reference for clinical diagnosis of hypertension. The application of deep learning to the classification of pulse waves in Traditional Chinese Medicine (TCM) related to hypertensive target organ damage (TOD) is hindered by challenges such as low classification accuracy and inadequate generalization performance. To address these challenges, we introduce a lightweight transfer learning model named MobileNetV2SCP. This model transforms time-domain pulse waves into 36-dimensional frequency-domain waveform feature maps and establishes a dedicated pre-training network based on these maps to enhance the learning capability for small samples. To improve global feature correlation, we incorporate a novel fusion attention mechanism (SAS) into the inverted residual structure, along with the utilization of 3 × 3 convolutional layers and BatchNorm layers to mitigate model overfitting. The proposed model is evaluated using cross-validation results from 805 cases of pulse waves associated with hypertensive TOD. The assessment metrics, including Accuracy (92.74 %), F1-score (91.47 %), and Area Under Curve (AUC) (97.12 %), demonstrate superior classification accuracy and generalization performance compared to various state-of-the-art models. Furthermore, this study investigates the correlations between time-domain and frequency-domain features in pulse waves and their classification in hypertensive TOD. It analyzes key factors influencing pulse wave classification, providing valuable insights for the clinical diagnosis of TOD. [ABSTRACT FROM AUTHOR]

Accurately predicting air pollutants, especially in urban areas with well-defined spatial structures, is crucial. Over the past decade, machine learning techniques have been widely used to forecast urban air quality. However, traditional machine learning approaches have limitations in accuracy and interpretability for predicting pollutants. In this study, we propose a convolutional neural network (CNN) model to predict the spatial distribution of CO concentration in Nanjing urban area at 10 m resolution. Our model incorporates various factors as input, such as building height, topography, emissions, and is trained against the outputs simulated by the parallelized large-eddy simulation model (PALM). The PALM model has 48 different scenarios that varied in emissions, wind speeds, and wind directions. The results display a strong consistency between the two models. Furthermore, we evaluate the performance of our model using a 10-fold cross-validation and out-of-sample cross-validation approach. This yields a robust correlation (with both R2 > 0.8) and a low RMSE between the CO predicted by the PALM and CNN models, which demonstrates the generalization capability of our CNN model. The CNN can extract crucial features from the resulted weight contribution map. This map indicates that the CO concentration at a location is more influenced by nearby buildings and emissions than distant ones. The interpretable patterns uncovered by our model are related to neighborhood effects, wind speeds, directions, and the impact of orientation on urban CO distribution. The model also shows high prediction accuracy (R > 0.8) when applied to another city. Overall, the integration of our CNN framework with the PALM model enhances the accuracy of air quality predictions, while enabling a fluid dynamic laws interpretation, providing effective tools for air quality management. [Display omitted] • Deep learning mimics physical air quality models and reduces computational costs. • Model results show robust accuracy and generalization capabilities. • Weight map improves the deep learning model interpretability. • Deep learning models can be generalized to different cities. [ABSTRACT FROM AUTHOR]

Highlights • A novel robust HDR color image watermarking method is presented. • The Tucker decomposition is used to divide the HDR color image along the three color channels so that feature maps are extracted for watermark embedding to balance the distortion of each channel. • The first feature map consists of RGB correlations, which is transformed by using Schur to build the stable size relation between pair-wise coefficients to obtain robustness. • The experimental results show the proposed method can resist most of TMOs effectively. Abstract In order to protect the copyright of high dynamic range (HDR) images, a novel watermarking method based on feature map extraction is proposed. Since the HDR image provides wide dynamic ranges of visible luminance, the correlations of the three RGB channels are much stronger than those of neighbor pixels. In order to preserve strong relationships of the three color channels, the Tucker decomposition is used to divide the HDR color image along the three color channels so that three feature maps are extracted. The first feature map of the three feature maps is employed to embed watermark for robustness, because it contains most of the energies in the HDR image. The first feature map is divided into non-overlapping blocks, and the Schur decomposition is operated on each block to generate a unitary matrix. Since the texture or edge remains after the tone mapping operation (TMO) of the HDR image, the stable size relation between the pair-wise coefficients in the unitary matrix is established to embed watermark. The experimental results show that the proposed method can efficiently resist different TMOs and common image attacks, outperforming other existing HDR image watermarking methods. [ABSTRACT FROM AUTHOR]

The use of Convolutional Neural Networks (CNNs) in image analysis and recognition paved the way for long-term visual place recognition. The transferable power of generic descriptors extracted at different layers of off-the-shelf CNNs has been successfully exploited in various visual place recognition scenarios. In this paper we tackle this problem by extracting the full output of an intermediate layer and building an image descriptor of lower dimensionality by omitting the activation of filters corresponding to environmental changes. Thus, we are able to increase the robustness of the cross-season visual place recognition. We test our approach on the Nordland dataset, the biggest and the most challenging dataset up to date, where the included four seasons induce great illumination and appearance changes. The experiments show that our new approach can significantly increase, up to 14%, the baseline (single-image search) performance of deep features. [ABSTRACT FROM AUTHOR]

In recent years, convolutional neural networks have become increasingly important in the field of machine learning, especially for computer vision. However, deep network models are difficult to deploy on hardware-constrained devices because of huge number of parameters, storage requirements and computational cost. This paper proposes a pruning feature maps method to delete redundant feature information in the deep network. Thus, it can simplify network structure at the same time reduce the computational complexity and speed up the operation. We first define a small chi-square supervised set. The feature maps of this set and the training set are extracted. Then, two variance matrices are constructed. The differences between the variance matrices are then used to establish chi-square distances. Through continuous experiments, the optimal position threshold is set, and the feature maps corresponding to the channels below the position threshold are pruned. Experiment performances show that the method can reduce network redundancy, storage space and network complexity totally up to 70%. This is accomplished without appreciably diminishing the network's accuracy. In the worst case, the accuracy difference for images classification by using simplified network and network before simplification was less than 0.4%. The use of a small pre-trained network also speeds up the network training. Together, these improvements constitute an important step towards the effective implementation of CNNs on constrained devices. [ABSTRACT FROM AUTHOR]

In theory, kernel support vector machines (SVMs) can be reformulated to linear SVMs. This reformulation can speed up SVM classifications considerably, in particular, if the number of support vectors is high. For the widely-used Gaussian radial basis function (RBF) kernel, however, this theoretical fact is impracticable because the reproducing kernel Hilbert space (RKHS) of this kernel has infinite dimensionality. Therefore, we derive a finite-dimensional approximative feature map, based on an orthonormal basis of the kernel’s RKHS, to enable the reformulation of Gaussian RBF SVMs to linear SVMs. We show that the error of this approximative feature map decreases with factorial growth if the approximation quality is linearly increased. Experimental evaluations demonstrated that the approximative feature map achieves considerable speed-ups (about 18-fold on average), mostly without losing classification accuracy. Therefore, the proposed approximative feature map provides an efficient SVM evaluation method with minimal loss of precision. [ABSTRACT FROM AUTHOR]

• A vision monitoring system for weld geometry of laser keyhole welding is proposed. • A high-accuracy and robust Multi-task CNN model is proposed and validated. • The proposed monitoring system can obtain the weld depth and width in real time. This paper presents a low-cost, robust, in-situ monitoring system for weld geometry that can achieve multi-task prediction. First, the system uses a low-cost CCD camera to monitor the melt pool in the laser keyhole welding process. Then, the proposed novel multi-task convolutional neural network (Multi-task CNN) model is used to simultaneously complete the two prediction tasks of weld depth and width. Furthermore, the learning process of the Multi-task CNN model is explored using a visual feature map approach and the robustness of the model is demonstrated. Compared with Support Vector Machine, K-Nearest Neighbor, Bayesian Ridge, Decision Tree, the proposed Multi-task CNN model has the highest prediction accuracy. The model predicts a mean absolute percentage error (MAPE, relative to ground truth) of 3.0% and 1.9% for weld depth and width. The in-situ monitoring results show that the system can achieve accurate predictions, and the average time-consuming of the system is 23.35 ms. [ABSTRACT FROM AUTHOR]

The ability to automatically classify files based on their low-level, short-range structures is of particular importance in computer forensics. We report a study on the automatic learning of file classification using byte sub-stream kernels that capture these low-level structures. We automatically discover byte-level patterns in a file by extracting a byte sequence feature map and use a suffix trie data structure to efficiently store and manipulate the feature map. Using the feature map we compute the spectrum kernel and, together with a support vector machine classifier algorithm, we are able to efficiently categorize a variety of different system and application file types. Experiments have provided good file classification performance results. [Copyright &y& Elsevier]

Relationships among cortical feature maps may influence the ability of the early visual system to sense the orientation and position of a line stimulus. Several combinations of orientation maps and visuotopic maps were studied. The neurons were characterized by their receptive fields in the space of stimulus parameters and a stochastical response. Fisher information was calculated for each stimulus parameter and the average bound on the variance was used as the measure of error of parameter estimation. We conclude that the distortions of the visuotopic map suggested by the experimentally measured statistics result in decreased accuracy of the parameter estimation. [Copyright &y& Elsevier]

• Direct utilization of feature maps from a pre-trained VGG-16 extracts the sewing stitches successfully without additional training. • CNN feature map-based broken stitch detection method is proposed. • The proposed approach is invariant to orientation and detects actual defects with 92.3% accuracy. • Conditions for real-time detection to real production sites are investigated. The inspection of sewing defects is an essential step in the quality assurance of garment manufacturing. Although traditional automated defect detection applications have shown good performance, these methods are usually configured with handcrafted features designed by a human operator. Recently, deep learning methods that include Convolutional Neural Networks (CNNs) have demonstrated excellent performance in a wide variety of computer-vision applications. To take advantage of the CNN's feature representation, the direct utilization of feature maps from the convolutional layers as universal feature descriptors has been studied. In this paper, we propose a sewing defect detection method using a CNN feature map extracted from the initial layers of a pre-trained VGG-16 to detect a broken stitch from a captured image of a sewing operation. To assess the effectiveness of the proposed method, experiments were conducted on a set of sewing images, including normal images, their synthetic defects, and rotated images. As a result, the proposed method detected true defects with 92.3% accuracy. Moreover, additional conditions for computing devices and deep learning libraries were investigated to reduce the computing time required for real-time computation. Using a general and cheap single-board computer with resizing the image and utilizing a lightweight deep learning library, the computing time was 0.22 s. The results confirm the feasibility of the proposed method's performance as an appropriate manufacturing technology for garment production. [ABSTRACT FROM AUTHOR]

While the identification of sand type helps naturally approximate physical and mechanical properties, it is challenging to judge sand types without prior information. This study attempts to identify the sand type in 2D grayscale images by using convolutional neural networks (CNNs). Six different sand samples with high geometric similarity were selected, and individual particle images were taken. Three pretrained networks (VGGNet, ResNet, and Inception) were implemented for retraining with parameter fine-tuning. The results show that most round and irregularly shaped sands are well classified with higher accuracy than sand samples with intermediate shape parameters. Additionally, it is confirmed that the feature maps obtained from multiple layers of trained CNNs sufficiently include the image characteristics of each sand particle. Misclassified particles are mostly found where the shape parameters distributions overlap. Higher accuracy is achieved by using grayscale images for training than using binary images. It implies that a better prediction can be produced when both surface texture and boundary morphology are concurrently trained. This study suggests the strong possibility of classifying sand types and further estimating soil properties only with images. • Convolutional neural networks can satisfactorily classify multiple sand types. • The Inception architecture outperforms the VGG and ResNet architectures in the classification task. • Both particle shape and surface texture are the keys to improving the accuracy. [ABSTRACT FROM AUTHOR]

• Chicken sounds were intercepted from complex noise. • A new method CSCNN was proposed in this paper. • CSCNN used to detect avian influenza-infected chickens via chicken sound. • The recognition accuracy is high to 97.43%. The modern poultry industry is large-scale and breeding-intensive, making the spread of disease in poultry easier, faster and more harmful. Avian influenza (AI) is the most important disease in poultry, and prevention and detection of avian influenza in poultry is a focus of scientific research and the poultry industry. In this paper, a new sound recognition method, the chicken sound convolutional neural network (CSCNN), is proposed for detection of chickens with avian influenza. According to the spectral differences in environmental noise, chicken behaviour noise and chicken sound, a method was designed to extract the chicken sound from complex sound data. Four features of the chicken sounds were calculated and combined into feature maps, including Logfbank, Mel Frequency Cepstrum Coefficient (MFCC), MFCC Delta and MFCC Delta-Delta. Finally, the sounds of healthy chickens and chickens with avian influenza were recognized using CSCNN. In the experiment, the recognition accuracies of CSCNN via spectrogram (CSCNN-S) were 93.01%, 95.05%, and 97.43% on the 2nd, 4th, and 6th day after injection with the H9N2 virus, and the recognition accuracies of CSCNN with feature mapping (CSCNN-F) were 89.79%, 93.56%, and 95.84%, respectively. The experimental results show that the method proposed in this paper can be used to quickly and effectively detect avian influenza-infected chickens via chicken sound. [ABSTRACT FROM AUTHOR]