Search

With the promotion of artificial intelligence in agriculture and the popularization of plug tray seedling-raising technology, seedling raising and transplanting have become the most popular planting modes. Miss-seeding is one of the most serious problems affecting seedling raising and transplanting. It not only affects the germination rate of seeds but also reduces the utilization rate of the plug tray. The experimental analysis of traditional machine vision-based miss-seeding showed that because of uneven lighting, the plug tray was wrongly identified as a seed under bright light, but the seeds in the dark were not easy to identify. When using the seeding area to identify seeds and noise, sweet corn seeds in a small area can be easily screened out. This paper proposes a method using the ResNet network with an attention mechanism to solve the above-mentioned problems. In this paper, the captured image was segmented into the images of a single plug tray, and a residual attention network was built; the detection scheme of miss-seeding was also converted into a dichotomous picture recognition task. This paper demonstrates that the residual attention network can effectively recognize and detect the seed images of sweet corn with very high accuracy. The results of the experiment showed that the average accuracy of this recognition model was 98%. The feature visualization method was used to analyze the features, further proving the effectiveness of the classification method of plug tray seedlings.

Military object detection from Unmanned Aerial Vehicle (UAV) reconnaissance images faces challenges, including lack of image data, images with poor quality, and small objects. In this work, we simulate UAV low-altitude reconnaissance and construct the UAV reconnaissance image tank database UAVT-3. Then, we improve YOLOv5 and propose UAVT-YOLOv5 for object detection of UAV images. First, data augmentation of blurred images is introduced to improve the accuracy of fog and motion-blurred images. Secondly, a large-scale feature map together with multi-scale feedback is added to improve the recognition ability of small objects. Thirdly, we optimize the loss function by increasing the loss penalty of small objects and classes with fewer samples. Finally, the anchor boxes are optimized by clustering the ground truth object box of UAVT-3. The feature visualization technique Class Action Mapping (CAM) is introduced to explore the mechanisms of the proposed model. The experimental results of the improved model evaluated on UAVT-3 show that the mAP reaches 99.2%, an increase of 2.1% compared with YOLOv5, the detection speed is 40 frames per second, and data augmentation of blurred images yields an mAP increase of 20.4% and 26.6% for fog and motion blur images detection. The class action maps show the discriminant region of the tanks is the turret for UAVT-YOLOv5.

We investigated whether the use of technological tools can effectively help in manipulating the increasing volume of audio data available through the use of long field recordings. We also explored whether we can address, by using these recordings and tools, audio data analysis, feature extraction and determine predominant patterns in the data. Similarly, we explored whether we can visualize feature clusters in the data and automatically detect sonic events. Our focus was primarily on enhancing the importance of natural-urban hybrid habitats within cities, which benefit communities in various ways, specifically through the natural soundscapes of these habitats that evoke memories and reinforce a sense of belonging for inhabitants. The loss of sonic heritage can be a precursor to the extinction of biodiversity within these habitats. By quantifying changes in the soundscape of these habitats over long periods of time, we can collect relevant information linked to this eventual loss. In this respect, we developed two approaches. The first was the comparison among habitats that progressively changed from natural to urban. The second was the optimization of the field recordings' labeling process. This was performed with labels corresponding to the annotations of classes of sonic events and their respective start and end times, including events temporarily superimposed on one another. We compared three habitats over time by using their sonic characteristics collected in field conditions. Comparisons of sonic similarity or dissimilarity among patches were made based on the Jaccard coefficient and uniform manifold approximation and projection (UMAP). Our SEDnet model achieves a F1-score of 0.79 with error rate 0.377 and with the area under PSD-ROC curve of 71.0. In terms of computational efficiency, the model is able to detect sound events from an audio file in a time of 14.49 s. With these results, we confirm the usefulness of the methods used in this work for the process of labeling field recordings. [ABSTRACT FROM AUTHOR]

Nowadays, most deep-learning-based bearing fault diagnosis methods are studied under the condition of steady speed, while the performance of these models cannot be fully played under time-varying conditions. Therefore, in order to facilitate the practical application of a deep learning model in bearing fault diagnosis, a vibration–speed fusion network is proposed, which utilizes a transformer with a self-attention module to extract vibration features and utilizes a sparse autoencoder (SAE) network to extract sparse features from speed pulse signal. The vibration–speed fusion network enables the efficient fusion of different signals in a high-dimensional vector space with a high degree of model interpretability, without additional signal processing steps. After tuning the hyperparameters of the network, the key segments of the bearing’s time-domain vibration signals can be optimally extracted, the network performance is much better than traditional deep learning methods, and the classification accuracy can reach 95.18% and 99.85% on the two public bearing datasets from the Xi’an Jiaotong University and the University of Ottawa.

The fingerprint is an important biological feature of the human body, which contains abundant biometric information. At present, the academic exploration of fingerprint gender characteristics is generally at the level of understanding, and the standardization research is quite limited. A robust approach is presented in this article, Dense Dilated Convolution ResNet Autoencoder, to extract valid gender information from fingerprints. By replacing the normal convolution operations with the atrous convolution in the backbone, prior knowledge is provided to keep the edge details, and the global reception field can be extended. The results were explored from three aspects: (1) Efficiency of DDC-ResNet. We conducted experiments using a combination of 6 typical automatic feature extractors with 9 classifiers for a total of 54 combinations are evaluated in our dataset; the experimental results show that the combination of methods we used achieved an average accuracy of 96.5%, with a classification accuracy of 97.52% for males and 95.48% for females, which outperformed the other experimental combinations. (2) The effect of the finger. The results showed that the right ring finger was the most effective for finger classification by gender. (3) The effect of specific features. We used the Class Activating Mapping method to plot fingerprint concentration thermograms, which allowed us to infer that fingerprint epidermal texture features are related to gender. The results demonstrated that autoencoder networks are a powerful method for extracting gender-specific features to help hide the privacy information of the user’s gender contained in the fingerprint. Our experiments also identified three levels of features in fingerprints that are important for gender differentiation, including loops and whorls shape, bifurcations shape, and line shapes.

Event-related potential (ERP) is bioelectrical activity that occurs in the brain in response to specific events or stimuli, reflecting the electrophysiological changes in the brain during cognitive processes. ERP is important in cognitive neuroscience and has been applied to brain-computer interfaces (BCIs). However, because ERP signals collected on the scalp are weak, mixed with spontaneous electroencephalogram (EEG) signals, and their temporal and spatial features are complex, accurate ERP detection is challenging. Compared to traditional neural networks, the capsule network (CapsNet) replaces scalar-output neurons with vector-output capsules, allowing the various input information to be well preserved in the capsules. In this study, we expect to utilize CapsNet to extract the discriminative spatial-temporal features of ERP and encode them in capsules to reduce the loss of valuable information, thereby improving the ERP detection performance for BCI. Therefore, we propose ERP-CapsNet to perform ERP detection in a BCI speller application. The experimental results on BCI Competition datasets and the Akimpech dataset show that ERP-CapsNet achieves better classification performances than do the state-of-the-art techniques. We also use a decoder to investigate the attributes of ERPs encoded in capsules. The results show that ERP-CapsNet relies on the P300 and P100 components to detect ERP. Therefore, ERP-CapsNet not only acts as an outstanding method for ERP detection, but also provides useful insights into the ERP detection mechanism. [ABSTRACT FROM AUTHOR]

Despite the great success of Generative Adversarial Networks (GANs) in synthesizing images, there lacks enough understanding of how photo-realistic images are generated from the layer-wise stochastic latent codes introduced in recent GANs. In this work, we show that highly-structured semantic hierarchy emerges in the deep generative representations from the state-of-the-art GANs like StyleGAN and BigGAN, trained for scene synthesis. By probing the per-layer representation with a broad set of semantics at different abstraction levels, we manage to quantify the causality between the layer-wise activations and the semantics occurring in the output image. Such a quantification identifies the human-understandable variation factors that can be further used to steer the generation process, such as changing the lighting condition and varying the viewpoint of the scene. Extensive qualitative and quantitative results suggest that the generative representations learned by the GANs with layer-wise latent codes are specialized to synthesize various concepts in a hierarchical manner: the early layers tend to determine the spatial layout, the middle layers control the categorical objects, and the later layers render the scene attributes as well as the color scheme. Identifying such a set of steerable variation factors facilitates high-fidelity scene editing based on well-learned GAN models without any retraining (code and demo video are available at https://genforce.github.io/higan). [ABSTRACT FROM AUTHOR]

In this work, we examined the color tuning of units in the hidden layers of AlexNet, VGG-16 and VGG-19 convolutional neural networks and their relevance for the successful recognition of an object. We first selected the patches for which the units are maximally responsive among the 1.2 M images of the ImageNet training dataset. We segmented these patches using a k-means clustering algorithm on their chromatic distribution. Then we independently varied the color of these segments, both in hue and chroma, to measure the unit's chromatic tuning. The models exhibited properties at times similar or opposed to the known chromatic processing of biological system. We found that, similarly to the most anterior occipital visual areas in primates, the last convolutional layer exhibited high color sensitivity. We also found the gradual emergence of single to double opponent kernels. Contrary to cells in the visual system, however, these kernels were selective for hues that gradually transit from being broadly distributed in early layers, to mainly falling along the blue-orange axis in late layers. In addition, we found that the classification performance of our models varies as we change the color of our stimuli following the models' kernels properties. Performance was highest for colors the kernels maximally responded to, and images responsible for the activation of color sensitive kernels were more likely to be mis-classified as we changed their color. These observations were shared by all three networks, thus suggesting that they are general properties of current convolutional neural networks trained for object recognition. [ABSTRACT FROM AUTHOR]

Feature visualization engineering of convolutional neural network is a basic research project in deep learning. The working principle of network, the image features extracted from network and classification basis of images can be revealed, by visualization the extracted features. In this paper, the de‐convolution and CAM method are used to visualize array defect features extracted by CNN. We find that the low‐layer network of CNN is unselective, and its main function is to separate all objects in the picture from their background, this is why the low‐layer networks' parameters usually do not been trained in transfer learning. Instead, the selectivity is slowly emerging in high‐layer networks and only the object features related to classification are preserved. At the same time, we find that the classification of array defects is not only related to itself, but also to its background, this can also explain why array defect classification is more difficult. On the other hand, this paper verifies the importance of array defect attributes in defect classification by counting the accuracy of network identification after modified the defect attributes, and a defect recognition disturbance rate index was defined to quantify the dependence of defect classification results on its attributes. Finally, the images edited by defect attributes with high disturbance rate are added to the training set, and the defect recognition accuracy is improved by 3% ~ 5%. This method can be used to judge the importance of each array defect attribute, find out the crucial attributes and pinpoint the completeness of training data in CNN classification. At the same time, it can explain the classification principle of CNN and provide necessary guidance and help for attribute image collection and CNN classification accuracy improvement. [ABSTRACT FROM AUTHOR]

Endocrine-disrupting chemicals (EDCs) pose significant environmental and health risks due to their potential to interfere with nuclear receptors (NRs), key regulators of physiological processes. Despite the evident risks, the majority of existing research narrows its focus on the interaction between compounds and the individual NR target, neglecting a comprehensive assessment across the entire NR family. In response, this study assembled a comprehensive human NR dataset, capturing 49,244 interactions between 35,467 unique compounds and 42 NRs. We introduced a cross-attention network framework, "CatNet", innovatively integrating compound and protein representations through cross-attention mechanisms. The results showed that CatNet model achieved excellent performance with an area under the receiver operating characteristic curve (AUC ROC) = 0.916 on the test set, and exhibited reliable generalization on unseen compound-NR pairs. A distinguishing feature of our research is its capacity to expand to novel targets. Beyond its predictive accuracy, CatNet offers a valuable mechanistic perspective on compound-NR interactions through feature visualization. Augmenting the utility of our research, we have also developed a graphical user interface, empowering researchers to predict chemical binding to diverse NRs. Our model enables the prediction of human NR-related EDCs and shows the potential to identify EDCs related to other targets. [Display omitted] • A novel deep learning model, CatNet, is provided to identify potential EDCs. • CatNet can be expanded to novel targets and identify EDCs targeting other proteins. • CatNet with cross-attention mechanisms enhances the model interpretability. • The development of GUI enhances accessibility, facilitating its use by the public. [ABSTRACT FROM AUTHOR]

Abstract Deep neural networks have become a standard framework for image analytics. Besides the traditional applications, such as object classification and detection, the latest studies have started to expand the scope of the applications to include artworks. However, popular art forms, such as comics, have been ignored in this trend. This study investigates visual features for comic classification using deep neural networks. An effective input format for comic classification is first defined, and a convolutional neural network is used to classify comic images into eight different artist categories. Using a publicly available dataset, the trained model obtains a mean F1 score of 84% for the classification. A feature visualization technique is also applied to the trained classifier, to verify the internal visual characteristics that succeed in classification. The experimental result shows that the visualized features are significantly different from those of general object classification. This work represents one of the first attempts to examine the visual characteristics of comics using feature visualization, in terms of comic author classification with deep neural networks.

As a deep learning model, a deep belief network (DBN) consists of multiple restricted Boltzmann machines (RBMs). Based on DBN, many intelligent fault diagnosis methods are proposed. However, these methods seldom considered the appearance of new working conditions during the operation of real machines. Varying working conditions lead to a change of feature distributions and finally result in low diagnosis accuracies. Therefore, we propose a distribution-invariant DBN (DIDBN) to learn distribution-invariant features directly from raw vibration data. DIDBN consists of a locally connected RBM (LCRBM) layer, a fully connected RBM layer, and an RBM layer with a mean discrepancy maximum (MDM-RBM). The LCRBM layer is designed with a locally connected structure. By proposing MDM, the MDM-RBM layer is able to obtain features that have close distributions under varying working conditions. Followed by a softmax classifier, DIDBN is able to recognize faults. The proposed method is applied to two diagnosis cases. Results verify that DIDBN is able to learn distribution-invariant features and achieve higher diagnosis accuracies than recently proposed methods. Moreover, the reason why DIDBN is able to learn distribution-invariant features is explained by visualizing the feature learning process. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) have attained human-level performance on dozens of challenging tasks via an end-to-end deep learning strategy. Deep learning allows data representations that have multiple levels of abstraction; however, it does not explicitly provide any insights into the internal operations of DNNs. Deep learning's success is appealing to neuroscientists not only as a method for applying DNNs to model biological neural systems but also as a means of adopting concepts and methods from cognitive neuroscience to understand the internal representations of DNNs. Although general deep learning frameworks, such as PyTorch and TensorFlow, could be used to allow such cross-disciplinary investigations, the use of these frameworks typically requires high-level programming expertise and comprehensive mathematical knowledge. A toolbox specifically designed as a mechanism for cognitive neuroscientists to map both DNNs and brains is urgently needed. Here, we present DNNBrain, a Python-based toolbox designed for exploring the internal representations of DNNs as well as brains. Through the integration of DNN software packages and well-established brain imaging tools, DNNBrain provides application programming and command line interfaces for a variety of research scenarios. These include extracting DNN activation, probing and visualizing DNN representations, and mapping DNN representations onto the brain. We expect that our toolbox will accelerate scientific research by both applying DNNs to model biological neural systems and utilizing paradigms of cognitive neuroscience to unveil the black box of DNNs.

Deep neural networks (DNNs) have attained human-level performance on dozens of challenging tasks via an end-to-end deep learning strategy. Deep learning allows data representations that have multiple levels of abstraction; however, it does not explicitly provide any insights into the internal operations of DNNs. Deep learning's success is appealing to neuroscientists not only as a method for applying DNNs to model biological neural systems but also as a means of adopting concepts and methods from cognitive neuroscience to understand the internal representations of DNNs. Although general deep learning frameworks, such as PyTorch and TensorFlow, could be used to allow such cross-disciplinary investigations, the use of these frameworks typically requires high-level programming expertise and comprehensive mathematical knowledge. A toolbox specifically designed as a mechanism for cognitive neuroscientists to map both DNNs and brains is urgently needed. Here, we present DNNBrain, a Python-based toolbox designed for exploring the internal representations of DNNs as well as brains. Through the integration of DNN software packages and well-established brain imaging tools, DNNBrain provides application programming and command line interfaces for a variety of research scenarios. These include extracting DNN activation, probing and visualizing DNN representations, and mapping DNN representations onto the brain. We expect that our toolbox will accelerate scientific research by both applying DNNs to model biological neural systems and utilizing paradigms of cognitive neuroscience to unveil the black box of DNNs. [ABSTRACT FROM AUTHOR]

Research explaining the behavior of convolutional neural networks (CNNs) has gained a lot of attention over the past few years. Although many visualization methods have been proposed to explain network predictions, most fail to provide clear correlations between the target output and the features extracted by convolutional layers. In this work, we define a concept, i.e., class-discriminative feature groups, to specify features that are extracted by groups of convolutional kernels correlated with a particular image class. We propose a detection method to detect class-discriminative feature groups and a visualization method to highlight image regions correlated with particular output and to interpret class-discriminative feature groups intuitively. The experiments showed that the proposed method can disentangle features based on image classes and shed light on what feature groups are extracted from which regions of the image. We also applied this method to visualize "lost" features in adversarial samples and features in an image containing a non-class object to demonstrate its ability to debug why the network failed or succeeded. • Class-discriminative feature visualization method for convolutional neural networks. • Detecting contributive features by Shapley-based attributions. • Decomposing features into groups to simplify visualizations. • Visualizing features with attributions to represent influences of features. • Application of visualizing lost features in adversarial samples. [ABSTRACT FROM AUTHOR]

Deep learning approaches, especially convolutional neural network (CNN) models, have achieved excellent performances in vibrational spectral analysis. The critical drawback of the CNN approach is the lack of interpretation, and it is regarded as a black box. Interpreting the learning mechanism of chemometric models is critical for intuitive understanding and further application. In this study, an interpretable CNN model with a global average pooling layer is presented for Raman and mid-infrared spectral data analysis. A class activation mapping (CAM)-based approach is leveraged to visualize the active variables in the whole spectrum. The visualization of active variables shows a discriminative pattern in which the most contributed variables peaked around theoretical chemical characteristic bands. The visualization of the feature maps by three convolutional layers demonstrates the data transformation pipeline and how the CNN model hierarchically extracts informative spectral features. The first layer acts as a Savitzky-Golay filter and learns spectral shape characteristics, while the second layer learns enhanced patterns from typical spectral peaks on a few correlated variables. The third layer shows stable activations on critical spectral peaks. A partial least squares - linear discriminant analysis (PLS-LDA) model is presented for comparison on classification accuracy and model interpretation. The CNN model yields mean classification accuracies of 99.01 and 100% for E. coli and meat datasets on the test set, while the PLS-LDA models obtain accuracies of 98.83 and 100%. Both the CNN and PLS-LDA models demonstrate stable patterns on active variables while CNN models are more stable than PLS-LDA models on classification performances for various dataset partitions with Monte-Carlo cross-validation. Image 1 • Identify active variables learned by a CNN model for spectral analysis. • Illustrate the feature representations via different convolutional layers. • Evaluate the model reliability with Monte-Carlo cross-validation. [ABSTRACT FROM AUTHOR]

Developmental dysplasia of the hip is a medical term representing the hip joint instability that appears mainly in infants. The assessment metric of physician is based on the femoral head coverage rate, which needs to segment the femoral head area in 2D ultrasound images. In this paper, we propose an approach to automatically segment the femoral head. The proposed method consists of two parts, firstly, mean filtering, morphological processing and least squares operation are used to detect the ilium and acetabular bone baseline to coarsely obtain the region of interest of the femoral head, then followed by an improved fully convolutional neural network named FNet which integrates the convolution encoder–decoder architecture, pooling indices and residual connection operation for more accurate segmentation. FNet is trained in a cascaded way, which can help the network learn more features with a limited dataset and thus further improve the segmentation performance. Experimental results show that the proposed method achieved an average dice, recall and IoU value of 0.946, 0.937 and 0.897. Moreover, the features learned by convolutional layers are visualized to demonstrate that FNet can focus on significant features, which is helpful to restore the contour of the femoral head more precisely. In conclusion, the proposed method is capable of segmenting femoral head accurately and guiding the diagnosis of developmental dysplasia of the hip. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Beijing University of Technology is the property of Journal of Beijing University of Technology, Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

In the Phalaenopsis industry, the growth vigor of seedlings when reaching their minimum growth time of vegetative cultivation plays an important role in the subsequent production chain and the final economic profits. The current manual assessment taking place in the commercial large-scale greenhouse is time-consuming and labor-intensive. Related studies based on RGB image for plant growth assessment relied on extracting hand-crafted features from images, affecting the effectiveness and generalization ability of machine learning models. In this study, the Convolutional Neural Network (CNN) was employed to explore its feasibility in assessing the growth vigor of Phalaenopsis aphrodite seedlings grown in the greenhouse in an end-to-end manner. Seedling images were collected in the greenhouse conditions with complex image background. Baseline models on the greenhouse dataset were established using different CNN architectures (VGG, ResNet, Inception-v3) coupled with various training mechanisms (training from scratch, fine-tuning, feature extraction), in which fine-tuning achieved the best classification results. Considering the target task of morphological classification for individual greenhouse seedlings with complex image background, to further boost model performance, additional seedlings images were acquired in controlled laboratory conditions. The segmented laboratory images were used to assist in model learning, namely building the augmented models. Two approaches were adopted, achieving an overall improvement in the testing F1-score of 0.03-0.05 compared with baseline models. The VGG model with augmentation method II achieved the highest performance in this study (F1-score of 0.997 on the test set), its feature maps were also visualized. In higher-level feature maps, regions of the target seedling were activated while filtering out most background including leaves from adjacent seedlings, proving the effective morphology characterizing using CNN for greenhouse seedlings. The overall results demonstrated the potential of deep learning models for image-based assessment of growth vigor for Phalaenopsis aphrodite seedlings and maybe other kinds of plants in greenhouse conditions. [ABSTRACT FROM AUTHOR]

This paper presents a video object segmentation method which jointly uses motion boundary and convolutional neural network (CNN)-based class-level maps to carry out the co-segmentation of the frames. The key characteristic of the proposed approach is a combination of those two sources of information to create initial object and background regions. These regions are employed within the co-segmentation energy function. The motion boundary map detects the areas which contain the object movement, and the CNN-based class saliency map determines the regions with more impact on acquiring the correct network classification. The proposed approach can be implemented on unconstrained natural videos which include changes in an object's appearance, rapidly moving background, object deformation in non-rigid moving, rapid camera motion and even the existence of a static object. Experimental results on two challenging datasets (i.e., Davis and SegTrackv2 datasets) demonstrate the competitive performance of the proposed method compared with the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Copyright of Journal of National University of Defense Technology / Guofang Keji Daxue Xuebao is the property of NUDT Press and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

We demonstrate a recognition and feature visualization method that uses a deep convolutional neural network for Raman spectrum analysis. The visualization is achieved by calculating important regions in the spectra from weights in pooling and fully-connected layers. The method is first examined for simple Lorentzian spectra, then applied to the spectra of pharmaceutical compounds and numerically mixed amino acids. We investigate the effects of the size and number of convolution filters on the extracted regions for Raman-peak signals using the Lorentzian spectra. It is confirmed that the Raman peak contributes to the recognition by visualizing the extracted features. A near-zero weight value is obtained at the background level region, which appears to be used for baseline correction. Common component extraction is confirmed by an evaluation of numerically mixed amino acid spectra. High weight values at the common peaks and negative values at the distinctive peaks appear, even though the model is given one-hot vectors as the training labels (without a mix ratio). This proposed method is potentially suitable for applications such as the validation of trained models, ensuring the reliability of common component extraction from compound samples for spectral analysis. Image 1 • A recognition and trained feature visualization that uses a CNN are demonstrated. • Raman peak extraction and near-zero feature value at background region are obtained. • Common component and extraordinary peak extraction from mixed spectra are confirmed. [ABSTRACT FROM AUTHOR]

This article presents a sea-sky-line detection algorithm in a sea-sky environment for unmanned surface vehicles. Obstacle detection is a vital branch for unmanned surface vehicles on the ocean. Because of the specificity and complexity of the marine navigation environment, we first apply semantic segmentation for marine images. The complete marine scene is divided into sky area, middle mixture area, and seawater area before sea-sky-line detection. Segmenting the marine environment is beneficial for narrowing the obstacle search area, accelerating the rate of obstacle detection, and improving detection accuracy. Therefore, a fast, robust, and accurate sea-sky image segmentation method is urgently required. Therefore, we present a method that lies in a probabilistic graphical model for segmenting marine images. The Gaussian mixture model is introduced as the probability distribution model for the marine image. The sky, middle mixture, and seawater areas are generated by three Gaussian models. The expectation--maximization algorithm is utilized to maximize the log-likelihood function, and the parameters of the Gaussian mixture probability density function that recover the marine image distribution are available after several iterations. Furthermore, to solve the problem of incorrect convergence direction caused by unsatisfactory initialization conditions, the gray level co-occurrence matrix is referenced to initialize the Gaussian components. The coarse segmentation results rely on the gray level co-occurrence matrix and are used to calculate the prior initialization parameters of Gaussian components and obtain the prior distribution information of marine images, which mitigates the harmful influence of poor initialization. The algorithm is tested on a data set consisting of the marine obstacle detection dataset (MODD) public data set and our collected images. The results on this data set demonstrate that the proposed method is more robust and that a superior initialization condition can effectively accelerate the convergence velocity of the iterative process for Gaussian components. [ABSTRACT FROM AUTHOR]

We investigated whether the use of technological tools can effectively help in manipulating the increasing volume of audio data available through the use of long field recordings. We also explored whether we can address, by using these recordings and tools, audio data analysis, feature extraction and determine predominant patterns in the data. Similarly, we explored whether we can visualize feature clusters in the data and automatically detect sonic events. Our focus was primarily on enhancing the importance of natural-urban hybrid habitats within cities, which benefit communities in various ways, specifically through the natural soundscapes of these habitats that evoke memories and reinforce a sense of belonging for inhabitants. The loss of sonic heritage can be a precursor to the extinction of biodiversity within these habitats. By quantifying changes in the soundscape of these habitats over long periods of time, we can collect relevant information linked to this eventual loss. In this respect, we developed two approaches. The first was the comparison among habitats that progressively changed from natural to urban. The second was the optimization of the field recordings’ labeling process. This was performed with labels corresponding to the annotations of classes of sonic events and their respective start and end times, including events temporarily superimposed on one another. We compared three habitats over time by using their sonic characteristics collected in field conditions. Comparisons of sonic similarity or dissimilarity among patches were made based on the Jaccard coefficient and uniform manifold approximation and projection (UMAP). Our SEDnet model achieves a F1-score of 0.79 with error rate 0.377 and with the area under PSD-ROC curve of 71.0. In terms of computational efficiency, the model is able to detect sound events from an audio file in a time of 14.49 s. With these results, we confirm the usefulness of the methods used in this work for the process of labeling field recordings.

Abstract The extensive deployment of surveillance cameras in public places, such as subway stations and shopping malls, necessitates automated visual-data processing approaches to match pedestrians across non-overlapping multiple cameras. However, due to the insufficient number of labeled training samples in real surveillance scene, it is difficult to train an effective deep neural network for cross-camera pedestrian recognition. Moreover, the cross-camera variation in viewpoint, illumination, and background makes the task even more challenging. To address these issues, in this paper we propose to transfer the parameters of a pre-trained network to our target network and then update the parameters adaptively using training samples from the target domain. More importantly, we develop new network structures that are specially tailored for cross-camera pedestrian recognition task, and implement a simple yet effective multi-level feature fusion method that yield more discriminative and robust features for pedestrian recognition. Specifically, rather than conventionally perform classification on the single-level feature of the last feature layer, we instead utilize multi-level feature by associating feature visualization with multi-level feature fusion. As another contribution, we have published our codes and extracted features to facilitate further research. Extensive experiments are conducted on WARD, PRID and MARS datasets, we show that the proposed method consistently outperforms state-of-the-arts. [ABSTRACT FROM AUTHOR]

This paper explores how well deep learning models trained on chest CT images can diagnose COVID-19 infected people in a fast and automated process. To this end, we adopted advanced deep network architectures and proposed a transfer learning strategy using custom-sized input tailored for each deep architecture to achieve the best performance. We conducted extensive sets of experiments on two CT image datasets, namely, the SARS-CoV-2 CT-scan and the COVID19-CT. The results show superior performances for our models compared with previous studies. Our best models achieved average accuracy, precision, sensitivity, specificity, and F1-score values of 99.4%, 99.6%, 99.8%, 99.6%, and 99.4% on the SARS-CoV-2 dataset, and 92.9%, 91.3%, 93.7%, 92.2%, and 92.5% on the COVID19-CT dataset, respectively. For better interpretability of the results, we applied visualization techniques to provide visual explanations for the models’ predictions. Feature visualizations of the learned features show well-separated clusters representing CT images of COVID-19 and non-COVID-19 cases. Moreover, the visualizations indicate that our models are not only capable of identifying COVID-19 cases but also provide accurate localization of the COVID-19-associated regions, as indicated by well-trained radiologists.

The broad availability of smartphones and Inertial Measurement Units in particular brings them into the focus of recent research. Inertial Measurement Unit data is used for a variety of tasks. One important task is the classification of the mode of transportation. In the first step, we present a deep-learning-based algorithm that combines long-short-term-memory (LSTM) layer and convolutional layer to classify eight different modes of transportation on the Sussex–Huawei Locomotion-Transportation (SHL) dataset. The inputs of our model are the accelerometer, gyroscope, linear acceleration, magnetometer, gravity and pressure values as well as the orientation information. In the second step, we analyze the contribution of each sensor modality to the classification score and to the different modes of transportation. For this analysis, we subtract the baseline confusion matrix from a confusion matrix of a network trained with a left-out sensor modality (difference confusion matrix) and we visualize the low-level features from the LSTM layers. This approach provides useful insights into the properties of the deep-learning algorithm and indicates the presence of redundant sensor modalities.

Understanding contributing neuron features is crucial to explaining convolutional neural network (CNN) decisions. The attribution research provides an effective way to detect contributing neuron features and numerically assign them attribution scores. However, a method to clearly and intuitively represent the implications hidden in neuron attributions is lacking. Attribution scores show the numerical importance of contributing neurons, but the meanings implied by these numerical scores are still not available. To mitigate this gap, we propose an optimization-based visualization method named attribution visualization, which enables an intuitive understanding of neuron attributions. Our approach is distinguished from existing visualization methods by its ability to produce noise-free result, i.e. , the ability to remove irrelevant regions from visualizations. We achieve this by introducing an optimizable mask into the visualization process and designing an objective function that simultaneously optimizes the area-constrained mask and visualization. Furthermore, we propose the fractal noise pyramid with diverse and natural frequency spectra as our mask perturbation technique which is key to removing unrelated noise in visualization. We implement several comparisons and user studies with other visual explanations to demonstrate the unique properties of our attribution visualization. We also apply our attribution visualization on two representative CNNs, showcasing its ability to intuitively understand contributing neuron features. [ABSTRACT FROM AUTHOR]

[Display omitted] • The 1D-CNN model was designed to predict the SSC of watermelon, and the optimal parameter combination of 1D-CNN was found. • Comparative performance of regression models (PLS, BPNN, ID-CNN) is reported. • The feature learning process of 1D-CNN was showed by visualization method. • A gradient-based CNN regression model feature visualization method was proposed. • The features learned by the regression models (PLS, BPNN, ID-CNN) in the NIR spectra were demonstrated. The soluble solids content（SSC）of fruits is an essential indicator of fruit taste and flavor. The near-infrared spectroscopy (NIRS) is widely used in fruit SSC detection. A common method to deal with such predictive modelling is the partial least-squares (PLSR). Wavelength selection algorithms are often employed to eliminate redundant wavelengths before constructing the model. However, the wavelength selection algorithms can not only increase the model complexity but also may lose valuable information. In this work, 1D-CNN was used to improve the prediction model of the watermelon SSC. First, the best parameter combination for 1D-CNN was found. The root mean square error of prediction (RMSEP) was 0.21 and the determination coefficient of prediction (R p 2) was 0.97. Then, the performance of PLSR and backpropagation neural network (BPNN) are compared. The results show that the R p 2 of 1D-CNN is improved by 14.1% and 6.6%, respectively. Finally, a gradient-based method for visualizing features of CNN regression models, namely regression activation mapping (RAM), was proposed. The results show that the features of 1D-CNN are consistent with the feature peaks of the spectrum, while the features of the BPNN and PLSR are not obvious. [ABSTRACT FROM AUTHOR]

The identification work based on inertial data is not limited by space, and has high flexibility and concealment. Previous research has shown that inertial data contains information related to behavior categories. This article discusses whether inertial data contains information related to human identity. The classification experiment, based on the neural network feature fitting function, achieves 98.17% accuracy on the test set, confirming that the inertial data can be used for human identification. The accuracy of the classification method without feature extraction on the test set is only 63.84%, which further indicates the need for extracting features related to human identity from the changes in inertial data. In addition, the research on classification accuracy based on statistical features discusses the effect of different feature extraction functions on the results. The article also discusses the dimensionality reduction processing and visualization results of the collected data and the extracted features, which helps to intuitively assess the existence of features and the quality of different feature extraction effects.

In recent years, convolutional neural networks (CNNs) have shown great success in the scene classification of computer vision images. Although these CNNs can achieve excellent classification accuracy, the discriminative ability of feature representations extracted from CNNs is still limited in distinguishing more complex remote sensing images. Therefore, we propose a unified feature fusion framework based on attention mechanism in this paper, which is called Deep Discriminative Representation Learning with Attention Map (DDRL-AM). Firstly, by applying Gradient-weighted Class Activation Mapping (Grad-CAM) algorithm, attention maps associated with the predicted results are generated in order to make CNNs focus on the most salient parts of the image. Secondly, a spatial feature transformer (SFT) is designed to extract discriminative features from attention maps. Then an innovative two-channel CNN architecture is proposed by the fusion of features extracted from attention maps and the RGB (red green blue) stream. A new objective function that considers both center and cross-entropy loss are optimized to decrease the influence of inter-class dispersion and within-class variance. In order to show its effectiveness in classifying remote sensing images, the proposed DDRL-AM method is evaluated on four public benchmark datasets. The experimental results demonstrate the competitive scene classification performance of the DDRL-AM approach. Moreover, the visualization of features extracted by the proposed DDRL-AM method can prove that the discriminative ability of features has been increased.

Graphical abstract Highlights • A feature visualization for CNNs method is exploited to improve co-segmentation. • Visualization is used as an auxiliary information to discriminate salient regions. • The visualization information is used as an extra energy term in the cost function. • Region occlusion sensitivity is introduced for feature visualization. • An adaptive strategy is designed to detect correct foreground/background regions. This paper addresses the co-segmentation problem using feature visualization for CNNs. Visualization is exploited as an auxiliary information to discriminate salient image regions (dubbed as “heat-regions”) from non-salient ones. Region occlusion sensitivity is proposed for feature visualization. The co-segmentation problem is formulated via a convex quadratic optimization which is initialized by the heat-regions. The information obtained through the visualization is considered as an extra energy term in the cost function. The results of the visualization demonstrate that there exist some heat-regions which are not productive in the co-segmentation. To detect helpful regions among them, an adaptive strategy in the form of an iterative algorithm is proposed according to the consistency among all images. Comparison experiments conducted on two benchmark datasets, iCoseg and MSRC, illustrate the superior performance of the proposed approach over state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

The recognition performance of visual recognition systems is highly dependent on extracting and representing the discriminative characteristics of image data. Convolutional neural networks (CNNs) have shown unprecedented success in a variety of visual recognition tasks due to their capability to provide in-depth representations exploiting visual image features of appearance, color, and texture. This paper presents a novel system for ear recognition based on ensembles of deep CNN-based models and more specifically the Visual Geometry Group (VGG)-like network architectures for extracting discriminative deep features from ear images. We began by training different networks of increasing depth on ear images with random weight initialization. Then, we examined pretrained models as feature extractors as well as fine-tuning them on ear images. After that, we built ensembles of the best models to further improve the recognition performance. We evaluated the proposed ensembles through identification experiments using ear images acquired under controlled and uncontrolled conditions from mathematical analysis of images (AMI), AMI cropped (AMIC) (introduced here), and West Pomeranian University of Technology (WPUT) ear datasets. The experimental results indicate that our ensembles of models yield the best performance with significant improvements over the recently published results. Moreover, we provide visual explanations of the learned features by highlighting the relevant image regions utilized by the models for making decisions or predictions.

In order to recognize indoor scenarios, we extract image features for detecting objects, however, computers can make some unexpected mistakes. After visualizing the histogram of oriented gradient (HOG) features, we find that the world through the eyes of a computer is indeed different from human eyes, which assists researchers to see the reasons that cause a computer to make errors. Additionally, according to the visualization, we notice that the HOG features can obtain rich texture information. However, a large amount of background interference is also introduced. In order to enhance the robustness of the HOG feature, we propose an improved method for suppressing the background interference. On the basis of the original HOG feature, we introduce a principal component analysis (PCA) to extract the principal components of the image colour information. Then, a new hybrid feature descriptor, which is named HOG–PCA (HOGP), is made by deeply fusing these two features. Finally, the HOGP is compared to the state-of-the-art HOG feature descriptor in four scenes under different illumination. In the simulation and experimental tests, the qualitative and quantitative assessments indicate that the visualizing images of the HOGP feature are close to the observation results obtained by human eyes, which is better than the original HOG feature for object detection. Furthermore, the runtime of our proposed algorithm is hardly increased in comparison to the classic HOG feature. [ABSTRACT FROM AUTHOR]

The present perspective discusses methods to detect concepts in internal representations (hidden layers) of deep neural networks (DNNs), such as network dissection, feature visualization, and testing with concept activation vectors (TCAV). I argue that these methods provide evidence that DNNs are able to learn non-trivial relations between concepts. However, the methods also require users to specify or detect concepts via (sets of) instances. This underdetermines the meaning of concepts, making the methods unreliable. The problem could be overcome, to some extent, by systematically combining the methods and by using synthetic datasets. The perspective also discusses how conceptual spaces-sets of concepts in internal representations-are shaped by a trade-off between predictive accuracy and compression. I argue that conceptual spaces are useful, or even necessary, to understand how concepts are formed in DNNs but that there is a lack of method for studying conceptual spaces., (© 2023 The Author(s).)

Medicinal plants have been used to treat diseases since ancient times. Plants used as raw materials for herbal medicine are known as medicinal plants [2]. The U. S. Forest Service estimates that 40% of pharmaceutical drugs in the Western world are derived from plants [1]. Seven thousand medical compounds are derived from plants in the modern pharmacopeia. Herbal medicine combines traditional empirical knowledge with modern science [2]. A medicinal plant is considered an important source of prevention against various diseases [2]. The essential medicine component is extracted from different parts of the plants [8]. In underdeveloped countries, people use medicinal plants as a substitute for medicine. There are various species of plants in the world. Herbs are one of them, which are of different shapes, colors, and leaves [5]. It is difficult for ordinary people to recognize these species of herbs. People use more than 50000 plants in the world for medicinal purposes. There are 8000 medicinal plants in India with evidence of medicinal properties [7]. Automatic classification of these plant species is important because it requires intensive domain knowledge to manually classify the proper species. Machine learning techniques are extensively used in classifying medicinal plant species from photographs, which is challenging but intriguing to academics. Artificial Neural Network classifiers' effective performance depends on the quality of the image dataset [4]. This article represents a medicinal plant dataset: an image dataset of ten different Bangladeshi plant species. Images of medicinal plant leaves were from various gardens, including the Pharmacy Garden at Khwaja Yunus Ali University and the Khwaja Yunus Ali Medical College & Hospital in Sirajganj, Bangladesh. Images were collected by taking pictures with high-resolution mobile phone cameras. Ten medicinal species, 500 images per species are included in the data set, namely, Nayantara ( Catharanthus roseus ), Pathor kuchi ( Kalanchoe pinnata ), Gynura procumbens ( Longevity spinach ), Bohera ( Terminalia bellirica ), Haritaki ( Terminalia chebula ), Thankuni ( Centella asiatica ), Neem ( Azadirachta indica ), Tulsi ( Ocimum tenniflorum ), Lemon grass ( Cymbopogon citratus ), and Devil backbone ( Euphorbia tithymaloides ). This dataset will benefit researchers applying machine learning and computer vision algorithms in several ways. For example, training and evaluation of machine learning models with this well-curated high-quality dataset, development of new computer vision algorithms, automatic medicinal plant identification in the field of botany and pharmacology for drug discovery and conservation, and data augmentation. Overall, this medicinal plant image dataset can provide researchers in the field of machine learning and computer vision with a valuable resource to develop and evaluate algorithms for plant phenotyping, disease detection, plant identification, drug development, and other tasks related to medicinal plants., Competing Interests: The authors declare that they have no known competing financial interest or personal relationships that could have appeared to influence the work reported in this paper., (© 2023 The Author(s).)

Surface defect detection is an important technique to realize product quality inspection. In this study, we develop an innovative multi-scale pooling convolutional neural network to accomplish high-accuracy steel surface defect classification. The model was built based on SqueezeNet, and experiments were carried out on the NEU noise-free and noisy testing set. Class activation map visualization proves that the multi-scale pooling model can accurately capture the defect location at multiple scales, and the defect feature information at different scales can complement and reinforce each other to obtain more robust results. Through T-SNE visualization analysis, it is found that the classification results of this model have large inter-class distance and small intra-class distance, indicating that this model has high reliability and strong generalization ability. In addition, the model is small in size (3MB) and runs at up to 130FPS on an NVIDIA 1080Ti GPU, making it suitable for applications with high real-time requirements., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2023 Fu, Zhang, Le, Li, Zhu, Niu, Chen, Sun and Shen.)

• One-dimensional CNN was designed to classify neuromuscular disorders based on needle electromyography (nEMG) signals • nEMGNet, a one-dimensional CNN, captures raw nEMG signal characteristics at high accuracy and short inference time • Heterogeneous data structure for each subject is mitigated by divide-and-vote algorithm • Learned features of nEMGNet resemble the typical signal characteristics of neuromuscular disorders Neuromuscular disorders are diseases that damage our ability to control body movements. Needle electromyography (nEMG) is often used to diagnose neuromuscular disorders, which is an electrophysiological test measuring electric signals generated from a muscle using an invasive needle. Characteristics of nEMG signals are manually analyzed by an electromyographer to diagnose the types of neuromuscular disorders, and this process is highly dependent on the subjective experience of the electromyographer. Contemporary computer-aided methods utilized deep learning image classification models to classify nEMG signals which are not optimized for classifying signals. Additionally, model explainability was not addressed which is crucial in medical applications. This study aims to improve prediction accuracy, inference time, and explain model predictions in nEMG neuromuscular disorder classification. This study introduces the nEMGNet, a one-dimensional convolutional neural network with residual connections designed to extract features from raw signals with higher accuracy and faster speed compared to image classification models from previous works. Next, the divide-and-vote (DiVote) algorithm was designed to integrate each subject's heterogeneous nEMG signal data structures and to utilize muscle subtype information for higher accuracy. Finally, feature visualization was used to identify the causality of nEMGNet diagnosis predictions, to ensure that nEMGNet made predictions on valid features, not artifacts. The proposed method was tested using 376 nEMG signals measured from 57 subjects between June 2015 to July 2020 in Seoul National University Hospital. The results from the three-class classification task demonstrated that nEMGNet's prediction accuracy of nEMG signal segments was 62.35%, and the subject diagnosis prediction accuracy of nEMGNet and the DiVote algorithm was 83.69 %, over 5-fold cross-validation. nEMGNet outperformed all models from previous works on nEMG diagnosis classification, and heuristic analysis of feature visualization results indicate that nEMGNet learned relevant nEMG signal characteristics. This study introduced nEMGNet and DiVote algorithm which demonstrated fast and accurate performance in predicting neuromuscular disorders based on nEMG signals. The proposed method may be applied in medicine to support real-time electrophysiologic diagnosis. [Display omitted] [ABSTRACT FROM AUTHOR]

We introduce algorithms to visualize feature spaces used by object detectors. Our method works by inverting a visual feature back to multiple natural images. We found that these visualizations allow us to analyze object detection systems in new ways and gain new insight into the detector's failures. For example, when we visualize the features for high scoring false alarms, we discovered that, although they are clearly wrong in image space, they often look deceptively similar to true positives in feature space. This result suggests that many of these false alarms are caused by our choice of feature space, and supports that creating a better learning algorithm or building bigger datasets is unlikely to correct these errors without improving the features. By visualizing feature spaces, we can gain a more intuitive understanding of recognition systems. [ABSTRACT FROM AUTHOR]