Search

Identifying indolent and aggressive prostate cancers is a critical problem for optimal treatment. The existing approaches of prostate cancer detection are facing challenges as the techniques rely on ground truth labels with limited accuracy, and histological similarity, and do not consider the disease pathology characteristics, and indefinite differences in appearance between the cancerous and healthy tissue lead to many false positive and false negative interpretations. Hence, this research introduces a comprehensive framework designed to achieve accurate identification and localization of prostate cancers, irrespective of their aggressiveness. This is accomplished through the utilization of a sophisticated multilevel bidirectional long short-term memory (Bi-LSTM) model. The pre-processed images are subjected to multilevel feature map-based U-Net segmentation, bolstered by ResNet-101 and a channel-based attention module that improves the performance. Subsequently, segmented images undergo feature extraction, encompassing various feature types, including statistical features, a global hybrid-based feature map, and a ResNet-101 feature map that enhances the detection accuracy. The extracted features are fed to the multilevel Bi-LSTM model, further optimized through channel and spatial attention mechanisms that offer the effective localization and recognition of complex structures of cancer. Further, the framework represents a promising approach for enhancing the diagnosis and localization of prostate cancers, encompassing both indolent and aggressive cases. Rigorous testing on a distinct dataset demonstrates the model's effectiveness, with performance evaluated through key metrics which are reported as 96.72%, 96.17%, and 96.17% for accuracy, sensitivity, and specificity respectively utilizing the dataset 1. For dataset 2, the model achieves the accuracy, sensitivity, and specificity values of 94.41%, 93.10%, and 94.96% respectively. These results surpass the efficiency of alternative methods. [ABSTRACT FROM AUTHOR]

Copyright of Transactions of Nanjing University of Aeronautics & Astronautics is the property of Editorial Department of Journal of Nanjing University of Aeronautics & Astronautics and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

To efficiently capture feature information in tasks of fine-grained image classification, this study introduces a new network model for fine-grained image classification, which utilizes a hybrid attention approach. The model is built upon a hybrid attention module (MA), and with the assistance of the attention erasure module (EA), it can adaptively enhance the prominent areas in the image and capture more detailed image information. Specifically, for tasks involving finegrained image classification, this study designs an attention module capable of applying the attention mechanism to both the channel and spatial dimensions. This highlights the important regions and key feature channels in the image, allowing for the extraction of distinct local features. Furthermore, this study presents an attention erasure module (EA) that can remove significant areas in the image based on the features identified; thus, shifting focus to additional feature details within the image and improving the diversity and completeness of the features. Moreover, this study enhances the pooling layer of ResNet50 to augment the perceptual region and the capability to extract features from the network’s less deep layers. For the objective of fine-grained image classification, this study extracts a variety of features and merges them effectively to create the final feature representation. To assess the effectiveness of the proposed model, experiments were conducted on three publicly available fine-grained image classification datasets: Stanford Cars, FGVC-Aircraft, and CUB-200–2011. The method achieved classification accuracies of 92.8, 94.0, and 88.2% on these datasets, respectively. In comparison with existing approaches, the efficiency of this method has significantly improved, demonstrating higher accuracy and robustness. [ABSTRACT FROM AUTHOR]

The analysis of remote-sensing images always requires the extraction of data about the aquatic environment. However, it might be challenging to identify any water surface since the backgrounds of water zones in remote sensing images are usually complicated structures and dense vegetation. Furthermore, less significant tributaries and edge data could not be accurately detected using traditional water detection methods. As a result, a spatial attention residual U-Net architecture is proposed to enhance the effectiveness of water body segmentation. The suggested approach reweights the feature representation spatially to obtain data on water features, using U-Net as the network architecture. The feature of the water zone is obtained using the residual block. It obtains more precise local position data for the water zone, which enhance edge segmentation accuracy. The spatial attention module retrieves, segregates, and combines the low-level information and high-level information as two discrete inputs in various dimensions. To effectively segregate the water region from the context, the spatial attention module combines spatial features with deep contextual information. The experiments are performed using satellite images of kaggle dataset aquatic bodies and a real-time dataset. The results of the experiments reveal 96% of accuracy that the suggested strategy out performs the existing models. [ABSTRACT FROM AUTHOR]

Green fruit detection is of great significance for estimating orchard yield and the allocation of water and fertilizer. However, due to the similar colors of green fruit and the background of images, the complexity of backgrounds and the difficulty in collecting green fruit datasets, there is currently no accurate and convenient green fruit detection method available for small datasets. The YOLO object detection model, a representative of the single-stage detection framework, has the advantages of a flexible structure, fast inference speed and excellent versatility. In this study, we proposed a model based on the improved YOLOv5 model that combined data augmentation methods to detect green fruit in a small dataset with a background of similar color. In the improved YOLOv5 model (YOLOv5-AT), a Conv-AT block and SA and CA blocks were designed to construct feature information from different perspectives and improve the accuracy by conveying local key information to the deeper layer. The proposed method was applied to green oranges, green tomatoes and green persimmons, and the mAPs were higher than those of other YOLO object detection models, reaching 84.6%, 98.0% and 85.1%, respectively. Furthermore, taking green oranges as an example, a mAP of 82.2% was obtained on the basis of retaining 50% of the original dataset (163 images), which was only 2.4% lower than that obtained when using 100% of the dataset (326 images) for training. Thus, the YOLOv5-AT model combined with data augmentation methods can effectively achieve accurate detection in small green fruit datasets under a similar color background. These research results could provide supportive data for improving the efficiency of agricultural production. [ABSTRACT FROM AUTHOR]

Human Activity Recognition (HAR) is crucial for the advancement of applications in smart environments, communication, IoT, security, and healthcare monitoring. Convolutional neural networks (CNNs) have made substantial contributions to human activity recognition (HAR). However, they frequently encounter difficulties in accurately discerning intricate human actions in real-time situations. This study aims to fill a significant research gap by incorporating the Convolutional Block Attention Module (CBAM) into CNN architectures. The goal is to improve the extraction of features from video sequences. The CBAM boosts the performance of the network by selectively prioritizing significant spatial and channel-wise data, resulting in improved detection of subtle activity patterns and increased stability in categorization. CBAM’s attention mechanism directly focuses and amplifies essential characteristics, which sets it apart from typical CNNs that lack a refined focus mechanism. This unique approach results in improved performance in behavior identification tests. The proposed CBAM-enhanced model has been extensively tested on benchmark datasets, yielding an accuracy of 94.23% on the HMDB51 dataset. It also achieved competitive results of 83.4% and 88.9% on the UCF-101 and UCF-50 datasets, respectively. However, there is still a lack of study in comprehending how CBAM adjusts to different CNN architectures and its suitability in varied HAR situations beyond controlled datasets. In future studies, it is imperative for researchers to investigate the integration of CBAM with other CNN frameworks, assess its efficacy in practical scenarios, and explore multi-modal sensor fusion techniques to enhance its reliability and utility. This study showcases the ability of CBAM to enhance HAR capabilities and also paves the way for future research to improve activity identification systems for wider and more practical uses.

Background: Acute Bilirubin Encephalopathy (ABE) is a major cause of infant mortality and disability, making early detection and treatment essential to prevent further progression and complications. Methods: To enhance the diagnostic capabilities of multi-modal Magnetic Resonance Imaging (MRI) for ABE, we proposed a deep learning model integrating an attention module (AM) with a central network (CentralNet). This model was tested on MRI data from 145 newborns diagnosed with ABE and 140 non-ABE newborns, utilizing both T1-weighted and T2-weighted images. Results: The findings indicated the following: (1) In single-modality experiments, the inclusion of AM significantly improved all the performance metrics compared to the models without AM. Specifically, for T1-weighted MRI, the accuracy was 0.639 ± 0.04, AUC was 0.682 ± 0.037, and sensitivity was 0.688 ± 0.09. For the T2-weighted images, the accuracy was 0.738 ± 0.039 and the AUC was 0.796 ± 0.025. (2) In multi-modal experiments, using T1 + T2 images, our model achieved the best accuracy of 0.845 ± 0.018, AUC of 0.913 ± 0.02, and sensitivity of 0.954 ± 0.069, compared to models without an AM and CentralNet. The specificity remained relatively stable, while the precision and F1 scores significantly increased, reaching 0.792 ± 0.048 and 0.862 ± 0.017, respectively. Conclusions: This study emphasizes the effectiveness of combining attention modules with CentralNet, significantly enhancing the accuracy of multi-modal MRI in classifying ABE. It presents a new perspective and possibility for the clinical application of multi-modal MRI imaging in the diagnosis of ABE. [ABSTRACT FROM AUTHOR]

Abstract Facial expression recognition (FER) is widely used in many fields. To further improve the accuracy of FER, this paper proposes a method based on double‐code LBP‐layer spatial‐attention network (DLSANet). The backbone model for the DLSANet is an emotion network (ENet), which is modified with a double‐code LBP (DLBP) layer and a spatial attention module. The DLBP layer is at the front of the first convolutional layer. More valuable features can be extracted by inputting the image processed by DLBP into convolutional layers. The JAFFE and CK+ datasets are used, which contain seven expressions: happiness, anger, disgust, neutral, fear, sadness, and surprise. The average of fivefold cross‐validation shows that DLSANet achieves a recognition accuracy of 93.81% and 98.68% on the JAFFE and CK+ datasets. The experiment reveals that the DLSANet can produce better classification results than state‐of‐the‐art methods.

Action recognition based on 3D technology is showing a trend of universal application in the field of kinesiology day by day. This paper utilizes 3D action recognition technology to develop an auxiliary teaching system suitable for college basketball teaching and training. The top-down method is used for human pose estimation to provide human skeleton characterization and shield obstacles such as background and illumination for action recognition technology. A new edge connection is added to the original graph convolution algorithm to form the human action recognition algorithm in this paper, and a spatial attention module is introduced to improve the accuracy of model action recognition. The system is equipped with two significant algorithms for human posture estimation and action recognition, and the final design is finished as an auxiliary system for college basketball teaching and training. The system can match the similarity between students’ movements and standard movements with a minimum of more than 80%, and the basketball scores of students are generally improved to more than 90 points after using the system.

To solve the problem of irregular and difficult segmentation of blood vessels in fundus retinal images, a multi-scale attention feature fusion network model based on a dual-decoder structure is proposed to achieve accurate segmentation of retinal blood vessels. The dual decoder branch network structure can reduce information loss. In the encoder, the multi-scale attention feature fusion module is designed to extract rich multi-scale features and the spatial attention module is combined to enhance the extraction of spatial context information and improve vascular recognition ability. Squeeze-and-excitation module is used to optimize aggregated features, suppress irrelevant feature channels and improve the comprehensive segmentation ability of the model. The experimental results on the DRIVE and CHASEDB1 data sets show that the recall rate can reach 0.841 1 and 0.855 1 respectively, making great progress compared with some advanced networks at present, with the maximum increase of 6.6% and 8.25% respectively. [ABSTRACT FROM AUTHOR]

The past years of COVID-19 have attracted researchers to carry out benchmark work in face mask detection. However, the existing work does not focus on the problem of reconstructing the face area behind the mask and completing the face that can be used for face recognition. In order to address this problem, in this work we have proposed a spatial attention module-based conditional generative adversarial network method that can generate plausible images of faces without masks by removing the face masks from the face region. The method proposed in this work utilizes a self-created dataset consisting of faces with three types of face masks for training and testing purposes. With the proposed method, an SSIM value of 0.91231 which is 3.89% higher and a PSNR value of 30.9879 which is 3.17% higher has been obtained as compared to the vanilla C-GAN method. [ABSTRACT FROM AUTHOR]

Abstract Aiming at the problem that the existing models have a poor segmentation effect on imbalanced data sets with small‐scale samples, a bilateral U‐Net network model with a spatial attention mechanism is designed. The model uses the lightweight MobileNetV2 as the backbone network for feature hierarchical extraction and proposes an Attentive Pyramid Spatial Attention (APSA) module compared to the Attenuated Spatial Pyramid module, which can increase the receptive field and enhance the information, and finally adds the context fusion prediction branch that fuses high‐semantic and low‐semantic prediction results, and the model effectively improves the segmentation accuracy of small data sets. The experimental results on the CamVid data set show that compared with some existing semantic segmentation networks, the algorithm has a better segmentation effect and segmentation accuracy, and its mIOU reaches 75.85%. Moreover, to verify the generality of the model and the effectiveness of the APSA module, experiments were conducted on the VOC 2012 data set, and the APSA module improved mIOU by about 12.2%.

Semantic segmentation has been one of the most powerful image-processing technologies for the more complex agricultural environment in the field. In this study, a UNet network model with improved spatial-coordinate attention (SPCA) was proposed to promote the efficiency of the refined network model for the multi-crop semantic segmentation. A new SPCA system was also designed to combine the spatial mechanism and coordinate attention (CA). Specifically, the maximum and average numbers were first determined among all the feature points on each feature layer in this same direction. The feature channels were stacked and adjusted with the maximum and average numbers. Another weight feature layer was then activated with the spatial feature points. Secondly, the location information was extracted along the two directions using a CA module after multiplying the feature layer by the main road. At the same time, the coefficient “r” was removed to retain the feature channel information without any extra calculation cost of the channel. Finally, the SPCA module was blended into the UNet model to verify the semantic segmentation, compared with the rest attention modules. The results show that there was an outstanding difference in the segmentation accuracy for the three semantic segmentation models at the different scales. Firstly, the figure of accuracy was much lower at the scale of 512×512 for the three models. The number of accuracies was about four percentage points higher at the scale of 1500×1500 for the three networks than those at the other two scales. The optimal combination was achieved in the UNet network model. Secondly, the highest accuracy was observed in the SPCA among the five modules. The mean intersection over union (MIoU), mean pixel accuracy (MPA), mean precision (MPrecision), and mean recall (MRecall) were all ahead of the four modules of squeeze-and-excitation networks (SENet), convolutional block attention module (CBAM), efficient channel attention (ECA), and CA, with the MIoU of 92.20%, the MPA of 95.97%, 1.16 and 0.76 percentage points higher than the highest precision of the MIoU and MPA of the four modules, respectively. The CA module ranked second after the SPCA, and the least was the ECA module. Furthermore, there was a very low accuracy gain before and after the replacement for the crop segmentation, because the ECA's 1D convolution was inferior to the full connection layer. The CBAM with spatial attention was added to focus on channel attention, indicating better segmentation accuracy. The CA also shared much better location information, compared with the spatial. Thirdly, the highest accuracy was found in the segmentation of flue-cured tobacco, the building was the second, and the corn was the last, when the crop was classified using the five modules. The intersection over union (IoU) and pixel accuracy (PA) in the SPCA were both superior to those in the other four. Relatively, the SPCA was beneficial in retaining the crop boundary information, clearly integral splitting picture, and fuzzy island. These mistakes were then removed to identify and classify the crop, due to the strong Robust Performance. Lastly, the four indexes of accuracy evaluation were performed the best in the UNet with SPCA. The highest accuracy of the MIoU was 1.3 percentage points higher than the rest. The UNet precision was ranked second. The lowest accuracy of the four evaluation indicators was found in the PSPNet. The SPCA-UNet was the commonly highest on the various accuracy levels of segmentation, each of which remained above about 93%. DeepLab v3+ had the lowest accuracy in the flue-cured tobacco. The lowest accuracy was PSPNet in the other three categories. Therefore, there was the most important information gained by the connection of multi-feature layers between the encoder and the decoder for the task of extracting multiple crops. This finding can provide a strong reference to extract the accuracy categories in the multiple crops, due to the excellent performance of the UNet using improved spatial-coordinate attention. Since the training scenario can be involved in the mobile terminal, the mobile platform can be expected to reduce the training time without significant loss of accuracy. [ABSTRACT FROM AUTHOR]

In order to address the timing problem, invalid data problem and deep feature extraction problem in the current deep learning based aero-engine remaining life prediction, a remaining life prediction method based on time-series residual neural networks is proposed. This method uses a combination of temporal feature extraction layer and deep feature extraction layer to build the network model. First, the temporal feature extraction layer with multi-head structure is used to extract rich temporal features; then, the spatial attention mechanism is applied to improve the weights of important data; finally, the deep feature extraction layer is used to process the deep features of the data. To verify the effectiveness of the proposed method, experiments are conducted on the C-MAPSS dataset provided by NASA. The experimental results show that the method proposed in this paper can make accurate predictions of the remaining service life under different sub-datasets and has outstanding performance advantages in comparison with other outstanding networks. [ABSTRACT FROM AUTHOR]

The environmental sound classification has great research significance in the fields of intelligent audio monitoring and other fields. A novel multi-frequency resolution (MFR) feature is proposed in this paper to solve the problem that the existing single frequency resolution time–frequency features of sound cannot effectively express the characteristics of multiple types of sound. The MFR feature is composed of three features with different frequency resolutions, which are compressed in varying degrees at the time dimension. This method not only has the effect of data augmentation but also can obtain more context information during the feature extraction. And the MFR features of Log-Mel Spectrogram, Cochleagram, and Constant Q-Transform are combined to form a multi-channel MFR feature. Also, a network named SacNet is built, which can effectively solve the problem that the time–frequency feature map of sound contains more invalid information. The basic structural unit of the SacNet consists of two parallel branches, one using depthwise separable convolution as the main feature extractor, and the other using spatial attention module to extract more effective information. Experiment results have demonstrated that the proposed method achieves the state-of-the-art accuracy of 97.5%, 93.1%, and 95.3% on three benchmark datasets of ESC10, ESC50, and UrbanSound8K respectively, which are increased by 3.3%, 0.5%, and 2.3% respectively compared with the previous advanced methods. [ABSTRACT FROM AUTHOR]

Facial expression recognition (FER) is widely used in many fields. To further improve the accuracy of FER, this paper proposes a method based on double‐code LBP‐layer spatial‐attention network (DLSANet). The backbone model for the DLSANet is an emotion network (ENet), which is modified with a double‐code LBP (DLBP) layer and a spatial attention module. The DLBP layer is at the front of the first convolutional layer. More valuable features can be extracted by inputting the image processed by DLBP into convolutional layers. The JAFFE and CK+ datasets are used, which contain seven expressions: happiness, anger, disgust, neutral, fear, sadness, and surprise. The average of fivefold cross‐validation shows that DLSANet achieves a recognition accuracy of 93.81% and 98.68% on the JAFFE and CK+ datasets. The experiment reveals that the DLSANet can produce better classification results than state‐of‐the‐art methods. [ABSTRACT FROM AUTHOR]

Vehicle Type Recognition (VTR) is a significant segment within the vehicle recognition field. It provides an alternative identification method aside from license plate recognition and vehicle make and model recognition. Most of the recent studies use Convolutional Neural Networks (CNNs) to perform VTR. However, the feature responses obtained from CNNs are not recalibrated based on saliency and this hinders the classification performance. In this study, we propose a Spatial Attention Module (SAM) that is compatible with the existing CNNs. We aim to exploit the spatial relationship between feature responses by scaling them according to their relative importance to increase classification accuracy. The results reveal the exceptional performance of SAM on Beijing Institute of Technology (BIT)-Vehicle, Stanford Cars and web-nature Comprehensive Cars (CompCarsWeb) with 96.92%, 84.48% and 95.96% accuracies, respectively. A qualitative inspection of the learned feature embedding suggests the high cohesivity of the features within the group. Furthermore, an ablation study is conducted to justify the hyperparameters of choice for SAM. SAM is also modular where it is highly compatible with other CNNs and it leads to considerable performance improvement. A comparison with existing attention modules suggests our proposal prevails in the VTR application. The inference times of 1 ms and 10 ms for CaffeNet-SAM and ResNet-SAM also make them suitable for real-time classification tasks.

With the extraordinary ability by representing human body structure as a spatial graph, graph convolution networks (GCNs) have progressed much in skeleton-based action recognition. However, these methods usually use a predefined graph to represent human body structure, which is limited to 1-hop neighborhood with fixed weights. To handle these limitations, we propose a fully attentional network (FAN). It dynamically computes the edge weights for each input sample between graph nodes, thus avoiding the predefined fixed weights. Besides, it could attend to distant nodes by calculating their edge weights based on their similarities, thus avoiding the limited spatial receptive field. As an effective feature extractor, FAN achieves new state-of-the-art accuracy on three large-scale datasets, i.e., NTU RGB+D 60, NTU RGB+D 120 and Kinetics Skeleton 400. Visualizations are given to verify that FAN could dynamically emphasize the graph nodes that are important in expressing an action.

Aiming at the problem that the existing models have a poor segmentation effect on imbalanced data sets with small‐scale samples, a bilateral U‐Net network model with a spatial attention mechanism is designed. The model uses the lightweight MobileNetV2 as the backbone network for feature hierarchical extraction and proposes an Attentive Pyramid Spatial Attention (APSA) module compared to the Attenuated Spatial Pyramid module, which can increase the receptive field and enhance the information, and finally adds the context fusion prediction branch that fuses high‐semantic and low‐semantic prediction results, and the model effectively improves the segmentation accuracy of small data sets. The experimental results on the CamVid data set show that compared with some existing semantic segmentation networks, the algorithm has a better segmentation effect and segmentation accuracy, and its mIOU reaches 75.85%. Moreover, to verify the generality of the model and the effectiveness of the APSA module, experiments were conducted on the VOC 2012 data set, and the APSA module improved mIOU by about 12.2%. [ABSTRACT FROM AUTHOR]

Background: Acute bilirubin encephalopathy (ABE) is a significant cause of neonatal mortality and disability. Early detection and treatment of ABE can prevent the further development of ABE and its long-term complications. Due to the limited classification ability of single-modal magnetic resonance imaging (MRI), this study aimed to validate the classification performance of a new deep learning model based on multimodal MRI images. Additionally, the study evaluated the effect of a spatial attention module (SAM) on improving the model's diagnostic performance in distinguishing ABE. Methods: This study enrolled a total of 97 neonates diagnosed with ABE and 80 neonates diagnosed with hyperbilirubinemia (HB, non-ABE). Each patient underwent three types of multimodal imaging, which included T1-weighted imaging (T1WI), T2-weighted imaging (T2WI), and an apparent diffusion coefficient (ADC) map. A multimodal MRI classification model based on the ResNet18 network with spatial attention modules was built to distinguish ABE from non-ABE. All combinations of the three types of images were used as inputs to test the model's classification performance, and we also analyzed the prediction performance of models with SAMs through comparative experiments. Results: The results indicated that the diagnostic performance of the multimodal image combination was better than any single-modal image, and the combination of T1WI and T2WI achieved the best classification performance (accuracy = 0.808 ± 0.069, area under the curve = 0.808 ± 0.057). The ADC images performed the worst among the three modalities' images. Adding spatial attention modules significantly improved the model's classification performance. Conclusion: Our experiment showed that a multimodal image classification network with spatial attention modules significantly improved the accuracy of ABE classification. [ABSTRACT FROM AUTHOR]

Most research efforts on image classification so far have been focused on medium-scale datasets. In addition, there exist other problems, such as difficulty in feature extraction and small sample size. In order to address above difficulties, this paper proposes a multi-perspective convolutional neural network model, which contains channel attention module and spatial attention module. The proposed modules derive attention graphs from channel dimension and spatial dimension respectively, then the input features are selectively learned according to the importance of the features. We explain how the gain in storage can be traded against a loss in accuracy and/or an increase in CPU cost. In addition, we give the interpretability of the model at multiple scales. Quantitative and qualitative experimental results demonstrate that the accuracy of our proposed model can be improved by up to 3.8% and outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Green fruit detection is of great significance for estimating orchard yield and the allocation of water and fertilizer. However, due to the similar colors of green fruit and the background of images, the complexity of backgrounds and the difficulty in collecting green fruit datasets, there is currently no accurate and convenient green fruit detection method available for small datasets. The YOLO object detection model, a representative of the single-stage detection framework, has the advantages of a flexible structure, fast inference speed and excellent versatility. In this study, we proposed a model based on the improved YOLOv5 model that combined data augmentation methods to detect green fruit in a small dataset with a background of similar color. In the improved YOLOv5 model (YOLOv5-AT), a Conv-AT block and SA and CA blocks were designed to construct feature information from different perspectives and improve the accuracy by conveying local key information to the deeper layer. The proposed method was applied to green oranges, green tomatoes and green persimmons, and the mAPs were higher than those of other YOLO object detection models, reaching 84.6%, 98.0% and 85.1%, respectively. Furthermore, taking green oranges as an example, a mAP of 82.2% was obtained on the basis of retaining 50% of the original dataset (163 images), which was only 2.4% lower than that obtained when using 100% of the dataset (326 images) for training. Thus, the YOLOv5-AT model combined with data augmentation methods can effectively achieve accurate detection in small green fruit datasets under a similar color background. These research results could provide supportive data for improving the efficiency of agricultural production.

Background: Acute Bilirubin Encephalopathy (ABE) is a major cause of infant mortality and disability, making early detection and treatment essential to prevent further progression and complications. Methods: To enhance the diagnostic capabilities of multi-modal Magnetic Resonance Imaging (MRI) for ABE, we proposed a deep learning model integrating an attention module (AM) with a central network (CentralNet). This model was tested on MRI data from 145 newborns diagnosed with ABE and 140 non-ABE newborns, utilizing both T1-weighted and T2-weighted images. Results: The findings indicated the following: (1) In single-modality experiments, the inclusion of AM significantly improved all the performance metrics compared to the models without AM. Specifically, for T1-weighted MRI, the accuracy was 0.639 ± 0.04, AUC was 0.682 ± 0.037, and sensitivity was 0.688 ± 0.09. For the T2-weighted images, the accuracy was 0.738 ± 0.039 and the AUC was 0.796 ± 0.025. (2) In multi-modal experiments, using T1 + T2 images, our model achieved the best accuracy of 0.845 ± 0.018, AUC of 0.913 ± 0.02, and sensitivity of 0.954 ± 0.069, compared to models without an AM and CentralNet. The specificity remained relatively stable, while the precision and F1 scores significantly increased, reaching 0.792 ± 0.048 and 0.862 ± 0.017, respectively. Conclusions: This study emphasizes the effectiveness of combining attention modules with CentralNet, significantly enhancing the accuracy of multi-modal MRI in classifying ABE. It presents a new perspective and possibility for the clinical application of multi-modal MRI imaging in the diagnosis of ABE.

Coronary artery segmentation is a pivotal field that has received increasing attention in recent years. However, this task remains challenging because of the inhomogeneous distributions of the contrast agent and dim light, resulting in noise, vascular breakages and small vessel losses in the obtained segmentation results. To acquire better automatic blood vessel segmentation results for coronary angiography images, a UNet-based segmentation network (SARC-UNet) is constructed for coronary artery segmentation; this approach is based on residual convolution and spatial attention. First, we use the low-light image enhancement (LIME) approach to increase the contrast and clarity levels of coronary angiography images. Then, we design two residual convolution fusion modules (RCFM1 and RCFM2) that can successfully fuse the local and global information of coronary images while also capturing the characteristics of finer-grained blood vessels, hence preventing the loss of tiny blood vessels in the segmentation findings. Finally, using a cascaded waterfall structure, we create a new location-enhanced spatial attention (LESA) mechanism that can efficiently improve the long-distance dependencies between coronary vascular pixel features, eradicating vascular ruptures and noise in the segmentation results. This article subjectively and objectively evaluates the experimental results. This method has performed well on five general indicators. Furthermore, it outperforms the connectivity indicators proposed in this article. This method can effectively segment blood vessels and obtain higher accuracy results. Numerous experiments have shown that the suggested method outperforms the state-of-the-art approaches, particularly in terms of vessel connectivity and small blood vessel segmentation. • To improve the phenomena of small vessel loss and vessel breakage in segmentation results, we propose a U-shaped network that combines residual convolution and improved spatial attention. • Two special modules with different structures were proposed to be used for encoding and decoding paths. This design can improve the network's ability to aggregate information. • To improve the network's long-distance dependence, location-enhanced spatial attention was proposed. [ABSTRACT FROM AUTHOR]

In recent years, with the extensive application of deep learning in images, the task of remote sensing image change detection has witnessed a significant improvement. Several excellent methods based on Convolutional Neural Networks and emerging transformer-based methods have achieved impressive accuracy. However, Convolutional Neural Network-based approaches have difficulties in capturing long-range dependencies because of their natural limitations in effective receptive field acquisition unless deeper networks are employed, introducing other drawbacks such as an increased number of parameters and loss of shallow information. The transformer-based methods can effectively learn the relationship between different regions, but the computation is inefficient. Thus, in this paper, a multi-scale feature aggregation via transformer (MFATNet) is proposed for remote sensing image change detection. To obtain a more accurate change map after learning the intra-relationships of feature maps at different scales through the transformer, MFATNet aggregates the multi-scale features. Moreover, the Spatial Semantic Tokenizer (SST) is introduced to obtain refined semantic tokens before feeding into the transformer structure to make it focused on learning more crucial pixel relationships. To fuse low-level features (more fine-grained localization information) and high-level features (more accurate semantic information), and to alleviate the localization and semantic gap between high and low features, the Intra- and Inter-class Channel Attention Module (IICAM) are integrated to further determine more convincing change maps. Extensive experiments are conducted on LEVIR-CD, WHU-CD, and DSIFN-CD datasets. Intersection over union (IoU) of 82.42 and F1 score of 90.36, intersection over union (IoU) of 79.08 and F1 score of 88.31, intersection over union (IoU) of 77.98 and F1 score of 87.62, respectively, are achieved. The experimental results achieved promising performance compared to certain previous state-of-the-art change detection methods. [ABSTRACT FROM AUTHOR]

Existing almost deep learning methods rely on a large amount of annotated data, so they are inappropriate for forest fire smoke detection with limited data. In this paper, a novel hybrid attention-based few-shot learning method, named Attention-Based Prototypical Network, is proposed for forest fire smoke detection. Specifically, feature extraction network, which consists of convolutional block attention module, could extract high-level and discriminative features and further decrease the false alarm rate resulting from suspected smoke areas. Moreover, we design a meta-learning module to alleviate the overfitting issue caused by limited smoke images, and the meta-learning network enables achieving effective detection via comparing the distance between the class prototype of support images and the features of query images. A series of experiments on forest fire smoke datasets and miniImageNet dataset testify that the proposed method is superior to state-of-the-art few-shot learning approaches. [ABSTRACT FROM AUTHOR]

Abstract Background Accurate and efficient measurement of the diameter at breast height (DBH) of individual trees is essential for forest inventories, ecological management, and carbon budget estimation. However, traditional diameter tapes are still the most widely used dendrometers in forest surveys, which makes DBH measurement time-consuming and labor-intensive. Automatic and easy-to-use devices for measuring DBH are highly anticipated in forest surveys. In this study, we present a handheld device for measuring the DBH of individual trees that uses digital cameras and laser ranging, allowing for an instant, automated, and contactless measurement of DBH. Results The base hardware of this device is a digital camera and a laser rangefinder, which are used to take a picture of the targeted tree trunk and record the horizontal distance between the digital camera and the targeted tree, respectively. The core software is composed of lightweight convolutional neural networks (CNNs), which includes an attention-focused mechanism for detecting the tree trunk to log the number of pixels between the edges. We also calibrated the digital camera to correct the distortion introduced by the lens system, and obtained the normalized focal length. Parameters including the horizontal distance between the digital camera and the targeted tree, number of pixels between the edges of the tree trunk, and normalized focal length were used to calculate the DBH based on the principles of geometrical optics. The measured diameter values, and the longitudes and latitudes of the measurement sites, were recorded in a text file, which is convenient to export to external flash disks. The field measurement accuracy test showed that the BIAS of the newly developed device was − 1.78 mm, and no significant differences were found between the measured diameter values and the true values (measured by the conventional tape). Furthermore, compared with most other image-based instruments, our device showed higher measurement accuracy. Conclusions The newly developed handheld device realized efficient, accurate, instant, and non-contact measurements of DBH, and the CNNs were proven to be successful in the detection of the tree trunk in our research. We believe that the newly developed device can fulfill the precision requirement in forest surveys, and that the application of this device can improve the efficiency of DBH measurements in forest surveys.

Human Activity Recognition (HAR) is crucial for the advancement of applications in smart environments, communication, IoT, security, and healthcare monitoring. Convolutional neural networks (CNNs) have made substantial contributions to human activity recognition (HAR). However, they frequently encounter difficulties in accurately discerning intricate human actions in real-time situations. This study aims to fill a significant research gap by incorporating the Convolutional Block Attention Module (CBAM) into CNN architectures. The goal is to improve the extraction of features from video sequences. The CBAM boosts the performance of the network by selectively prioritizing significant spatial and channel-wise data, resulting in improved detection of subtle activity patterns and increased stability in categorization. CBAM's attention mechanism directly focuses and amplifies essential characteristics, which sets it apart from typical CNNs that lack a refined focus mechanism. This unique approach results in improved performance in behavior identification tests. The proposed CBAM-enhanced model has been extensively tested on benchmark datasets, yielding an accuracy of 94.23% on the HMDB51 dataset. It also achieved competitive results of 83.4% and 88.9% on the UCF-101 and UCF-50 datasets, respectively. However, there is still a lack of study in comprehending how CBAM adjusts to different CNN architectures and its suitability in varied HAR situations beyond controlled datasets. In future studies, it is imperative for researchers to investigate the integration of CBAM with other CNN frameworks, assess its efficacy in practical scenarios, and explore multi-modal sensor fusion techniques to enhance its reliability and utility. This study showcases the ability of CBAM to enhance HAR capabilities and also paves the way for future research to improve activity identification systems for wider and more practical uses. [ABSTRACT FROM AUTHOR]

Background: Acute bilirubin encephalopathy (ABE) is a significant cause of neonatal mortality and disability. Early detection and treatment of ABE can prevent the further development of ABE and its long-term complications. Due to the limited classification ability of single-modal magnetic resonance imaging (MRI), this study aimed to validate the classification performance of a new deep learning model based on multimodal MRI images. Additionally, the study evaluated the effect of a spatial attention module (SAM) on improving the model’s diagnostic performance in distinguishing ABE. Methods: This study enrolled a total of 97 neonates diagnosed with ABE and 80 neonates diagnosed with hyperbilirubinemia (HB, non-ABE). Each patient underwent three types of multimodal imaging, which included T1-weighted imaging (T1WI), T2-weighted imaging (T2WI), and an apparent diffusion coefficient (ADC) map. A multimodal MRI classification model based on the ResNet18 network with spatial attention modules was built to distinguish ABE from non-ABE. All combinations of the three types of images were used as inputs to test the model’s classification performance, and we also analyzed the prediction performance of models with SAMs through comparative experiments. Results: The results indicated that the diagnostic performance of the multimodal image combination was better than any single-modal image, and the combination of T1WI and T2WI achieved the best classification performance (accuracy = 0.808 ± 0.069, area under the curve = 0.808 ± 0.057). The ADC images performed the worst among the three modalities’ images. Adding spatial attention modules significantly improved the model’s classification performance. Conclusion: Our experiment showed that a multimodal image classification network with spatial attention modules significantly improved the accuracy of ABE classification.

Chinese Herbal Slices (CHS) are critical components of Traditional Chinese Medicine (TCM); the accurate recognition of CHS is crucial for applying to medicine, production, and education. However, existing methods to recognize the CHS are mainly performed by experienced professionals, which may not meet vast CHS market demand due to time-consuming and the limited number of professionals. Although some automated CHS recognition approaches have been proposed, the performance still needs further improvement because they are primarily based on the traditional machine learning with hand-crafted features, resulting in relatively low accuracy. Additionally, few CHS datasets are available for research aimed at practical application. To comprehensively address these problems, we propose a combined channel attention and spatial attention module network (CCSM-Net) for efficiently recognizing CHS with 2-D images. The CCSM-Net integrates channel and spatial attentions, focusing on the most important information as well as the position of the information of CHS image. Especially, pairs of max-pooling and average pooling operations are used in the CA and SA module to aggregate the channel information of the feature map. Then, a dataset of 14,196 images with 182 categories of commonly used CHS is constructed. We evaluated our framework on the constructed dataset. Experimental results show that the proposed CCSM-Net indicates promising performance and outperforms other typical deep learning algorithms, achieving a recognition rate of 99.27%, a precision of 99.33%, a recall of 99.27%, and an F1-score of 99.26% with different numbers of CHS categories. [ABSTRACT FROM AUTHOR]

Concomitant with the recent advances in deep learning, automatic speech recognition and visual speech recognition (VSR) have received considerable attention. However, although VSR systems must identify speech from both frontal and profile faces in real-world scenarios, most VSR studies have focused solely on frontal face pictures. To address this issue, we propose an end-to-end sentence-level multi-view VSR architecture for faces captured from four different perspectives (frontal, 30°, 45°, and 60°). The encoder uses multiple convolutional neural networks with a spatial attention module to detect minor changes in the mouth patterns of similarly pronounced words, and the decoder uses cascaded local self-attention connectionist temporal classification to collect the details of local contextual information in the immediate vicinity, which results in a substantial performance boost and speedy convergence. To compare the performance of the proposed model for experiments on the OuluVS2 dataset, the dataset was divided into four different perspectives, and the obtained performance improvement was 3.31% (0°), 4.79% (30°), 5.51% (45°), 6.18% (60°), and 4.95% (mean), respectively, compared with the existing state-of-the-art performance, and the average performance improved by 9.1% compared with the baseline. Thus, the suggested design enhances the performance of multi-view VSR and boosts its usefulness in real-world applications. [ABSTRACT FROM AUTHOR]

Chinese Herbal Slices (CHS) are critical components of Traditional Chinese Medicine (TCM); the accurate recognition of CHS is crucial for applying to medicine, production, and education. However, existing methods to recognize the CHS are mainly performed by experienced professionals, which may not meet vast CHS market demand due to time-consuming and the limited number of professionals. Although some automated CHS recognition approaches have been proposed, the performance still needs further improvement because they are primarily based on the traditional machine learning with hand-crafted features, resulting in relatively low accuracy. Additionally, few CHS datasets are available for research aimed at practical application. To comprehensively address these problems, we propose a combined channel attention and spatial attention module network (CCSM-Net) for efficiently recognizing CHS with 2-D images. The CCSM-Net integrates channel and spatial attentions, focusing on the most important information as well as the position of the information of CHS image. Especially, pairs of max-pooling and average pooling operations are used in the CA and SA module to aggregate the channel information of the feature map. Then, a dataset of 14,196 images with 182 categories of commonly used CHS is constructed. We evaluated our framework on the constructed dataset. Experimental results show that the proposed CCSM-Net indicates promising performance and outperforms other typical deep learning algorithms, achieving a recognition rate of 99.27%, a precision of 99.33%, a recall of 99.27%, and an F1-score of 99.26% with different numbers of CHS categories.

In recent years, the research of deep learning has received extensive attention, and many breakthroughs have been made in various fields. On this basis, a neural network with the attention mechanism has become a research hotspot. In this paper, we try to solve the image classification task by implementing channel and spatial attention mechanism which improve the expression ability of neural network model. Different from previous studies, we propose an attention module consisting of channel attention module (CAM) and spatial attention module (SAM). The proposed module derives attention graphs from channel dimension and spatial dimension respectively, then the input features are selectively learned according to the importance of the features. Besides, this module is lightweight and can be easily integrated into image classification algorithms. In the experiment, we combine the deep residual network model with the attention module and the experimental results show that the proposed method brings higher image classification accuracy. The channel attention module adds weight to the signals on different convolution channels to represent the correlation. For different channels, the higher the weight, the higher the correlation which required more attention. The main function of spatial attention is to capture the most informative part in the local feature graph, which is a supplement to channel attention. We evaluate our proposed module based on the ImageNet-1K and Cifar-100 respectively. Through a large number of comparative experiments, our proposed model achieved outstanding performance. [ABSTRACT FROM AUTHOR]

Identifying indolent and aggressive prostate cancers is a critical problem for optimal treatment. The existing approaches of prostate cancer detection are facing challenges as the techniques rely on ground truth labels with limited accuracy, and histological similarity, and do not consider the disease pathology characteristics, and indefinite differences in appearance between the cancerous and healthy tissue lead to many false positive and false negative interpretations. Hence, this research introduces a comprehensive framework designed to achieve accurate identification and localization of prostate cancers, irrespective of their aggressiveness. This is accomplished through the utilization of a sophisticated multilevel bidirectional long short-term memory (Bi-LSTM) model. The pre-processed images are subjected to multilevel feature map-based U-Net segmentation, bolstered by ResNet-101 and a channel-based attention module that improves the performance. Subsequently, segmented images undergo feature extraction, encompassing various feature types, including statistical features, a global hybrid-based feature map, and a ResNet-101 feature map that enhances the detection accuracy. The extracted features are fed to the multilevel Bi-LSTM model, further optimized through channel and spatial attention mechanisms that offer the effective localization and recognition of complex structures of cancer. Further, the framework represents a promising approach for enhancing the diagnosis and localization of prostate cancers, encompassing both indolent and aggressive cases. Rigorous testing on a distinct dataset demonstrates the model's effectiveness, with performance evaluated through key metrics which are reported as 96.72%, 96.17%, and 96.17% for accuracy, sensitivity, and specificity respectively utilizing the dataset 1. For dataset 2, the model achieves the accuracy, sensitivity, and specificity values of 94.41%, 93.10%, and 94.96% respectively. These results surpass the efficiency of alternative methods., (© 2024. The Author(s) under exclusive licence to Society for Imaging Informatics in Medicine.)

In recent years, with the extensive application of deep learning in images, the task of remote sensing image change detection has witnessed a significant improvement. Several excellent methods based on Convolutional Neural Networks and emerging transformer-based methods have achieved impressive accuracy. However, Convolutional Neural Network-based approaches have difficulties in capturing long-range dependencies because of their natural limitations in effective receptive field acquisition unless deeper networks are employed, introducing other drawbacks such as an increased number of parameters and loss of shallow information. The transformer-based methods can effectively learn the relationship between different regions, but the computation is inefficient. Thus, in this paper, a multi-scale feature aggregation via transformer (MFATNet) is proposed for remote sensing image change detection. To obtain a more accurate change map after learning the intra-relationships of feature maps at different scales through the transformer, MFATNet aggregates the multi-scale features. Moreover, the Spatial Semantic Tokenizer (SST) is introduced to obtain refined semantic tokens before feeding into the transformer structure to make it focused on learning more crucial pixel relationships. To fuse low-level features (more fine-grained localization information) and high-level features (more accurate semantic information), and to alleviate the localization and semantic gap between high and low features, the Intra- and Inter-class Channel Attention Module (IICAM) are integrated to further determine more convincing change maps. Extensive experiments are conducted on LEVIR-CD, WHU-CD, and DSIFN-CD datasets. Intersection over union (IoU) of 82.42 and F1 score of 90.36, intersection over union (IoU) of 79.08 and F1 score of 88.31, intersection over union (IoU) of 77.98 and F1 score of 87.62, respectively, are achieved. The experimental results achieved promising performance compared to certain previous state-of-the-art change detection methods.

Background: Accurate and efficient measurement of the diameter at breast height (DBH) of individual trees is essential for forest inventories, ecological management, and carbon budget estimation. However, traditional diameter tapes are still the most widely used dendrometers in forest surveys, which makes DBH measurement time-consuming and labor-intensive. Automatic and easy-to-use devices for measuring DBH are highly anticipated in forest surveys. In this study, we present a handheld device for measuring the DBH of individual trees that uses digital cameras and laser ranging, allowing for an instant, automated, and contactless measurement of DBH. Results: The base hardware of this device is a digital camera and a laser rangefinder, which are used to take a picture of the targeted tree trunk and record the horizontal distance between the digital camera and the targeted tree, respectively. The core software is composed of lightweight convolutional neural networks (CNNs), which includes an attention-focused mechanism for detecting the tree trunk to log the number of pixels between the edges. We also calibrated the digital camera to correct the distortion introduced by the lens system, and obtained the normalized focal length. Parameters including the horizontal distance between the digital camera and the targeted tree, number of pixels between the edges of the tree trunk, and normalized focal length were used to calculate the DBH based on the principles of geometrical optics. The measured diameter values, and the longitudes and latitudes of the measurement sites, were recorded in a text file, which is convenient to export to external flash disks. The field measurement accuracy test showed that the BIAS of the newly developed device was − 1.78 mm, and no significant differences were found between the measured diameter values and the true values (measured by the conventional tape). Furthermore, compared with most other image-based instruments, our device showed higher measurement accuracy. Conclusions: The newly developed handheld device realized efficient, accurate, instant, and non-contact measurements of DBH, and the CNNs were proven to be successful in the detection of the tree trunk in our research. We believe that the newly developed device can fulfill the precision requirement in forest surveys, and that the application of this device can improve the efficiency of DBH measurements in forest surveys. [ABSTRACT FROM AUTHOR]

Concomitant with the recent advances in deep learning, automatic speech recognition and visual speech recognition (VSR) have received considerable attention. However, although VSR systems must identify speech from both frontal and profile faces in real-world scenarios, most VSR studies have focused solely on frontal face pictures. To address this issue, we propose an end-to-end sentence-level multi-view VSR architecture for faces captured from four different perspectives (frontal, 30°, 45°, and 60°). The encoder uses multiple convolutional neural networks with a spatial attention module to detect minor changes in the mouth patterns of similarly pronounced words, and the decoder uses cascaded local self-attention connectionist temporal classification to collect the details of local contextual information in the immediate vicinity, which results in a substantial performance boost and speedy convergence. To compare the performance of the proposed model for experiments on the OuluVS2 dataset, the dataset was divided into four different perspectives, and the obtained performance improvement was 3.31% (0°), 4.79% (30°), 5.51% (45°), 6.18% (60°), and 4.95% (mean), respectively, compared with the existing state-of-the-art performance, and the average performance improved by 9.1% compared with the baseline. Thus, the suggested design enhances the performance of multi-view VSR and boosts its usefulness in real-world applications.

To efficiently capture feature information in tasks of fine-grained image classification, this study introduces a new network model for fine-grained image classification, which utilizes a hybrid attention approach. The model is built upon a hybrid attention module (MA), and with the assistance of the attention erasure module (EA), it can adaptively enhance the prominent areas in the image and capture more detailed image information. Specifically, for tasks involving fine-grained image classification, this study designs an attention module capable of applying the attention mechanism to both the channel and spatial dimensions. This highlights the important regions and key feature channels in the image, allowing for the extraction of distinct local features. Furthermore, this study presents an attention erasure module (EA) that can remove significant areas in the image based on the features identified; thus, shifting focus to additional feature details within the image and improving the diversity and completeness of the features. Moreover, this study enhances the pooling layer of ResNet50 to augment the perceptual region and the capability to extract features from the network's less deep layers. For the objective of fine-grained image classification, this study extracts a variety of features and merges them effectively to create the final feature representation. To assess the effectiveness of the proposed model, experiments were conducted on three publicly available fine-grained image classification datasets: Stanford Cars, FGVC-Aircraft, and CUB-200-2011. The method achieved classification accuracies of 92.8, 94.0, and 88.2% on these datasets, respectively. In comparison with existing approaches, the efficiency of this method has significantly improved, demonstrating higher accuracy and robustness., Competing Interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest., (Copyright © 2024 Lu, Yang and Yang.)

This Letter presents an attention‐modulating network for video object segmentation that can well adapt its segmentation model to the annotated frame. Specifically, the authors first develop an efficient visual and spatial attention modulator to fast modulate the segmentation model to focus on the specific object of interest. Then they design a channel and spatial attention module and inject it into the segmentation model to further refine its feature maps. In addition, to fuse multi‐scale context information, they construct a feature pyramid attention module to further process the top layer feature maps, achieving better pixel‐level attention for the high‐level feature maps. Finally, to address the sample imbalance issue in training, they employ focal loss that can distinguish simple samples from the difficult ones to accelerate the convergence of network training. Extensive evaluations on DAVIS2017 dataset show that the proposed approach has achieved state‐of‐the‐art performance, outperforming the baseline OSMN by 3.6 and 5.4% in terms of IoU and F‐measure without fine‐tuning.

One essential step in radiotherapy treatment planning is the organ at risk of segmentation in Computed Tomography (CT). Many recent studies have focused on several organs such as the lung, heart, esophagus, trachea, liver, aorta, kidney, and prostate. However, among the above organs, the esophagus is one of the most difficult organs to segment because of its small size, ambiguous boundary, and very low contrast in CT images. To address these challenges, we propose a fully automated framework for the esophagus segmentation from CT images. The proposed method is based on the processing of slice images from the original three-dimensional (3D) image so that our method does not require large computational resources. We employ the spatial attention mechanism with the atrous spatial pyramid pooling module to locate the esophagus effectively, which enhances the segmentation performance. To optimize our model, we use group normalization because the computation is independent of batch sizes, and its performance is stable. We also used the simultaneous truth and performance level estimation (STAPLE) algorithm to reach robust results for segmentation. Firstly, our model was trained by k-fold cross-validation. And then, the candidate labels generated by each fold were combined by using the STAPLE algorithm. And as a result, Dice and Hausdorff Distance scores have an improvement when applying this algorithm to our segmentation results. Our method was evaluated on SegTHOR and StructSeg 2019 datasets, and the experiment shows that our method outperforms the state-of-the-art methods in esophagus segmentation. Our approach shows a promising result in esophagus segmentation, which is still challenging in medical analyses.

The automatic detection of aircrafts from SAR images is widely applied in both military and civil fields, but there are still considerable challenges. To address the high variety of aircraft sizes and complex background information in SAR images, a new fast detection framework based on convolution neural networks is proposed, which achieves automatic and rapid detection of aircraft with high accuracy. First, the airport runway areas are detected to generate the airport runway mask and rectangular contour of the whole airport are generated. Then, a new deep neural network proposed in this paper, named Efficient Weighted Feature Fusion and Attention Network (EWFAN), is used to detect aircrafts. EWFAN integrates the weighted feature fusion module, the spatial attention mechanism, and the CIF loss function. EWFAN can effectively reduce the interference of negative samples and enhance feature extraction, thereby significantly improving the detection accuracy. Finally, the airport runway mask is applied to the detected results to reduce false alarms and produce the final aircraft detection results. To evaluate the performance of the proposed framework, large-scale Gaofen-3 SAR images with 1 m resolution are utilized in the experiment. The detection rate and false alarm rate of our EWFAN algorithm are 95.4% and 3.3%, respectively, which outperforms Efficientdet and YOLOv4. In addition, the average test time with the proposed framework is only 15.40 s, indicating satisfying efficiency of automatic aircraft detection.

This Letter presents an attention‐modulating network for video object segmentation that can well adapt its segmentation model to the annotated frame. Specifically, the authors first develop an efficient visual and spatial attention modulator to fast modulate the segmentation model to focus on the specific object of interest. Then they design a channel and spatial attention module and inject it into the segmentation model to further refine its feature maps. In addition, to fuse multi‐scale context information, they construct a feature pyramid attention module to further process the top layer feature maps, achieving better pixel‐level attention for the high‐level feature maps. Finally, to address the sample imbalance issue in training, they employ focal loss that can distinguish simple samples from the difficult ones to accelerate the convergence of network training. Extensive evaluations on DAVIS2017 dataset show that the proposed approach has achieved state‐of‐the‐art performance, outperforming the baseline OSMN by 3.6 and 5.4% in terms of IoU and F‐measure without fine‐tuning. [ABSTRACT FROM AUTHOR]

The environmental sound classification has great research significance in the fields of intelligent audio monitoring and other fields. A novel multi-frequency resolution (MFR) feature is proposed in this paper to solve the problem that the existing single frequency resolution time-frequency features of sound cannot effectively express the characteristics of multiple types of sound. The MFR feature is composed of three features with different frequency resolutions, which are compressed in varying degrees at the time dimension. This method not only has the effect of data augmentation but also can obtain more context information during the feature extraction. And the MFR features of Log-Mel Spectrogram, Cochleagram, and Constant Q-Transform are combined to form a multi-channel MFR feature. Also, a network named SacNet is built, which can effectively solve the problem that the time-frequency feature map of sound contains more invalid information. The basic structural unit of the SacNet consists of two parallel branches, one using depthwise separable convolution as the main feature extractor, and the other using spatial attention module to extract more effective information. Experiment results have demonstrated that the proposed method achieves the state-of-the-art accuracy of 97.5%, 93.1%, and 95.3% on three benchmark datasets of ESC10, ESC50, and UrbanSound8K respectively, which are increased by 3.3%, 0.5%, and 2.3% respectively compared with the previous advanced methods., (© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2022, Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.)

• A light end-to-end dual-modality multi-scale pedestrian detection framework, which can achieve real-time detection speed, is introduced. • The ASPFF Net, which can adaptively extract features from different modalities from the spatial and pixel levels is proposed. • The proposed method is more competitive in terms of detection speed and has a lower miss rate on the reasonable night subset of the KAIST dataset than the other comparison methods. A pedestrian detector that uses visible and thermal infrared image pairs as the input has better detection performance than a detector that uses only visible image under challenging illumination conditions. With the aim to efficiently and effectively fuse complementary information from visible and thermal infrared images, this paper proposes an adaptive spatial pixel-level feature fusion network called the ASPFF Net, which can adaptively extract spatial pixel-level features from visible and infrared images for fusion. Specifically, first, two light networks with different weights are used to extract multi-scale features of visible and infrared images. Next, for features of the same scale but different modalities, the fusion weights of different spatial positions and pixels in the two feature maps are obtained by the spatial attention module (SAM) and pixel attention module (PAM). The original features of visible and infrared images are recalibrated by the fusion weights, and multi-scale fused feature layers are obtained. Finally, different scales of pedestrians are detected on the fused multi-scale feature layers. Compared with the other recent multispectral pedestrian detectors on the reasonable subset of the KAIST multispectral pedestrian detection dataset, the proposed detector is attractive in balancing speed and accuracy. The extensive experiments on the KAIST dataset demonstrate the effectiveness of the proposed method for the fusion of visible and infrared image in multispectral pedestrian detection. [ABSTRACT FROM AUTHOR]

One essential step in radiotherapy treatment planning is the organ at risk of segmentation in Computed Tomography (CT). Many recent studies have focused on several organs such as the lung, heart, esophagus, trachea, liver, aorta, kidney, and prostate. However, among the above organs, the esophagus is one of the most difficult organs to segment because of its small size, ambiguous boundary, and very low contrast in CT images. To address these challenges, we propose a fully automated framework for the esophagus segmentation from CT images. The proposed method is based on the processing of slice images from the original three-dimensional (3D) image so that our method does not require large computational resources. We employ the spatial attention mechanism with the atrous spatial pyramid pooling module to locate the esophagus effectively, which enhances the segmentation performance. To optimize our model, we use group normalization because the computation is independent of batch sizes, and its performance is stable. We also used the simultaneous truth and performance level estimation (STAPLE) algorithm to reach robust results for segmentation. Firstly, our model was trained by k-fold cross-validation. And then, the candidate labels generated by each fold were combined by using the STAPLE algorithm. And as a result, Dice and Hausdorff Distance scores have an improvement when applying this algorithm to our segmentation results. Our method was evaluated on SegTHOR and StructSeg 2019 datasets, and the experiment shows that our method outperforms the state-of-the-art methods in esophagus segmentation. Our approach shows a promising result in esophagus segmentation, which is still challenging in medical analyses. [ABSTRACT FROM AUTHOR]

• To mitigate the paucity of endoscopy data with time-synchronized 6 DoF ground truth pose, and high precision topographical information obtained from multiple ex-vivo organs and phantom colon, we have created a comprehensive EndoSLAM dataset recorded using multiple endoscopic cameras. • EndoSLAM dataset is available as an opensource and convenient to use for depth, pose estimation, simultaneous localization and mapping (SLAM), 3D map reconstruction and domain adaptation tasks. • We have also propounded Endo-SfMLearner as a spatial attention based monocular depth and pose estimation algorithm which is specifically optimized via hybrid loss function to deal with the problems in endoscopy images such as frequently changing enlightenment conditions and scale inconsistency between consecutive frames. Experimental analysis shows the applicability of Endo-SfMLearner on endoscopy images compared to the state-of-the-art methods.(Zhou, Brown, Snavely, Lowe, 2017) [Display omitted] Deep learning techniques hold promise to develop dense topography reconstruction and pose estimation methods for endoscopic videos. However, currently available datasets do not support effective quantitative benchmarking. In this paper, we introduce a comprehensive endoscopic SLAM dataset consisting of 3D point cloud data for six porcine organs, capsule and standard endoscopy recordings, synthetically generated data as well as clinically in use conventional endoscope recording of the phantom colon with computed tomography(CT) scan ground truth. A Panda robotic arm, two commercially available capsule endoscopes, three conventional endoscopes with different camera properties, two high precision 3D scanners, and a CT scanner were employed to collect data from eight ex-vivo porcine gastrointestinal (GI)-tract organs and a silicone colon phantom model. In total, 35 sub-datasets are provided with 6D pose ground truth for the ex-vivo part: 18 sub-datasets for colon, 12 sub-datasets for stomach, and 5 sub-datasets for small intestine, while four of these contain polyp-mimicking elevations carried out by an expert gastroenterologist. To verify the applicability of this data for use with real clinical systems, we recorded a video sequence with a state-of-the-art colonoscope from a full representation silicon colon phantom. Synthetic capsule endoscopy frames from stomach, colon, and small intestine with both depth and pose annotations are included to facilitate the study of simulation-to-real transfer learning algorithms. Additionally, we propound Endo-SfMLearner, an unsupervised monocular depth and pose estimation method that combines residual networks with a spatial attention module in order to dictate the network to focus on distinguishable and highly textured tissue regions. The proposed approach makes use of a brightness-aware photometric loss to improve the robustness under fast frame-to-frame illumination changes that are commonly seen in endoscopic videos. To exemplify the use-case of the EndoSLAM dataset, the performance of Endo-SfMLearner is extensively compared with the state-of-the-art: SC-SfMLearner, Monodepth2, and SfMLearner. The codes and the link for the dataset are publicly available at https://github.com/CapsuleEndoscope/EndoSLAM. A video demonstrating the experimental setup and procedure is accessible as Supplementary Video 1. [ABSTRACT FROM AUTHOR]

The automatic detection of aircrafts from SAR images is widely applied in both military and civil fields, but there are still considerable challenges. To address the high variety of aircraft sizes and complex background information in SAR images, a new fast detection framework based on convolution neural networks is proposed, which achieves automatic and rapid detection of aircraft with high accuracy. First, the airport runway areas are detected to generate the airport runway mask and rectangular contour of the whole airport are generated. Then, a new deep neural network proposed in this paper, named Efficient Weighted Feature Fusion and Attention Network (EWFAN), is used to detect aircrafts. EWFAN integrates the weighted feature fusion module, the spatial attention mechanism, and the CIF loss function. EWFAN can effectively reduce the interference of negative samples and enhance feature extraction, thereby significantly improving the detection accuracy. Finally, the airport runway mask is applied to the detected results to reduce false alarms and produce the final aircraft detection results. To evaluate the performance of the proposed framework, large-scale Gaofen-3 SAR images with 1 m resolution are utilized in the experiment. The detection rate and false alarm rate of our EWFAN algorithm are 95.4% and 3.3%, respectively, which outperforms Efficientdet and YOLOv4. In addition, the average test time with the proposed framework is only 15.40 s, indicating satisfying efficiency of automatic aircraft detection. [ABSTRACT FROM AUTHOR]