Search

Vegetation segmentation plays a crucial role in accurately monitoring and analyzing vegetation cover, growth patterns, and changes over time, which in turn contributes to environmental studies, land management, and assessing the impact of climate change. This study explores the potential of a multi-scale convolutional neural network (MSCNN) design for object classification, specifically focusing on vegetation detection. The MSCNN is designed to integrate multi-scale feature extraction and attention mechanisms, enabling the model to capture both fine and coarse vegetation patterns effectively. Moreover, the MSCNN architecture integrates multiple convolutional layers with varying kernel sizes (3 × 3, 5 × 5, and 7 × 7), enabling the model to extract features at different scales, which is vital for identifying diverse vegetation patterns across various landscapes. Vegetation detection is demonstrated using three diverse datasets: the CamVid dataset, the FloodNet dataset, and the multispectral RIT-18 dataset. These datasets present a range of challenges, including variations in illumination, the presence of shadows, occlusion, scale differences, and cluttered backgrounds, which are common in real-world scenarios. The MSCNN architecture allows for the integration of information from multiple scales, facilitating the detection of diverse vegetation types under varying conditions. The performance of the proposed MSCNN method is rigorously evaluated and compared against state-of-the-art techniques in the field. Comprehensive experiments showcase the effectiveness of the approach, highlighting its robustness in accurately segmenting and classifying vegetation even in complex environments. The results indicate that the MSCNN design significantly outperforms traditional methods, achieving a remarkable global accuracy and boundary F1 score (BF score) of up to 98%. This superior performance underscores the MSCNN’s capability to enhance vegetation detection in imagery, making it a promising tool for applications in environmental monitoring and land use management.

With the deep integration of psychology and artificial intelligence technology and other related technologies, eye control technology has achieved certain results at the practical application level. However, it is found that the accuracy of the current single-modal eye control technology is still not high, which is mainly caused by the inaccurate eye movement detection caused by the high randomness of eye movements in the process of human–computer interaction. Therefore, this study will propose an intent recognition method that fuses facial expressions and eye movement information and expects to complete an eye control method based on the fusion of facial expression and eye movement information based on the multimodal intent recognition dataset, including facial expressions and eye movement information constructed in this study. Based on the self-attention fusion strategy, the fused features are calculated, and the multi-layer perceptron is used to classify the fused features, so as to realize the mutual attention between different features, and improve the accuracy of intention recognition by enhancing the weight of effective features in a targeted manner. In order to solve the problem of inaccurate eye movement detection, an improved YOLOv5 model was proposed, and the accuracy of the model detection was improved by adding two strategies: a small target layer and a CA attention mechanism. At the same time, the corresponding eye movement behavior discrimination algorithm was combined for each eye movement action to realize the output of eye behavior instructions. Finally, the experimental verification of the eye–computer interaction scheme combining the intention recognition model and the eye movement detection model showed that the accuracy of the eye-controlled manipulator to perform various tasks could reach more than 95 percent based on this scheme.

Eye health has become a significant concern in recent years, given the rising prevalence of visual impairment resulting from various eye disorders and related factors. Global surveys suggest that approximately 2.2 billion individuals are visually impaired, with at least 1 billion affected by treatable diseases or ailments. Early detection, treatment, and screening for fundus diseases are crucial in addressing these challenges. In this study, we propose a novel segmentation model for retinal vascular delineation aimed at diagnosing diabetic retinopathy. The model integrates CBAM (Channel-Attention and Spatial-Attention) for enhanced feature representation, JPU (Joint Pyramid Upsampling) for multi-scale feature fusion, and transformer blocks for contextual understanding. Leveraging deep-learning techniques, our proposed model outperforms existing approaches in retinal vascular segmentation, like achieving a Mean IOU of 0.8047, Recall of 0.7254, Precision of 0.8492, F1 Score of 0.7824, and Specificity of 0.9892 for CHASEDB1 dataset. Extensive evaluations on benchmark datasets demonstrate its efficacy, highlighting its potential for automated diabetic retinopathy screening.

Integrating multi-modal data in natural language processing has opened new pathways for the enhancement of dialogue summarization. However, existing models often struggle to effectively synthesize textual, auditory, and visual inputs. This paper introduces a Modality-Enhanced Multi-Channel Attention Network (MEMA), a novel approach designed to optimize the integration and interaction of diverse modalities for dialogue summarization. MEMA leverages symmetrical embedding strategies to balance the integrity and distinctiveness of each modality, ensuring a harmonious interaction within the unified architecture. By maintaining symmetry in the processing flow, MEMA enhances the contextual richness and coherence of the generated summaries. Our model demonstrates superior performance on the Multi-modal Dialogue Summarization (MDS) dataset, particularly in generating contextually enriched abstract summaries. The results underscore MEMA’s potential to transform dialogue summarization by providing a more symmetrical and integrated understanding of multi-modal interactions, bridging the gap in multi-modal data processing, and setting a new standard for future summarization tasks.

Accurate pedestrian trajectory prediction is crucial in many fields. This requires the full use and learning of pedestrians’ social interactions, movements, and environmental information. In view of the current research on pedestrian trajectory prediction, wherein most of the pedestrian interaction information is explored from the level of overall interaction, this paper proposes the SISGAN model, which designs a social interaction module from the perspective of the target pedestrian, and takes four kinds of interaction information as the influencing factors of pedestrian interaction, so as to describe the influence mechanism of pedestrian–pedestrian interaction. In addition, in terms of environmental information, the index density of pedestrian historical trajectory in space is taken into account in the extraction of environmental information, which increases the potential correlation between environmental information and pedestrians. Finally, we integrate social interaction information and environmental information and make the final trajectory prediction based on GAN. Experiments on ETH and UCY datasets demonstrate the effectiveness of the SISGAN model proposed in this paper.

The human visual system can rapidly focus on prominent objects in complex scenes, significantly enhancing information processing efficiency. Salient object detection (SOD) mimics this biological ability, aiming to identify and segment the most prominent regions or objects in images or videos. This reduces the amount of data needed to process while enhancing the accuracy and efficiency of information extraction. In recent years, SOD has made significant progress in many areas such as deep learning, multi-modal fusion, and attention mechanisms. Additionally, it has expanded in real-time detection, weakly supervised learning, and cross-domain applications. Depth images can provide three-dimensional structural information of a scene, aiding in a more accurate understanding of object shapes and distances. In SOD tasks, depth images enhance detection accuracy and robustness by providing additional geometric information. This additional information is particularly crucial in complex scenes and occlusion situations. This survey reviews the substantial advancements in the field of RGB-Depth SOD, with a focus on the critical roles played by attention mechanisms and cross-modal fusion methods. It summarizes the existing literature, provides a brief overview of mainstream datasets and evaluation metrics, and quantitatively compares the discussed models.

Aspect-level sentiment analysis is a research focal point for natural language comprehension. An attention mechanism is a very important approach for aspect-level sentiment analysis, but it only fuses sentences from a semantic perspective and ignores grammatical information in the sentences. Graph convolutional networks (GCNs) are a better method for processing syntactic information; however, they still face problems in effectively combining semantic and syntactic information. This paper presents a sentiment-supported graph convolutional network (SSGCN). This SSGCN first obtains the semantic information of the text through aspect-aware attention and self-attention; then, a grammar mask matrix and a GCN are applied to preliminarily combine semantic information with grammatical information. Afterward, the processing of these information features is divided into three steps. To begin with, features related to the semantics and grammatical features of aspect words are extracted. The second step obtains the enhanced features of the semantic and grammatical information through sentiment support words. Finally, it concatenates the two features, thus enhancing the effectiveness of the attention mechanism formed from the combination of semantic and grammatical information. The experimental results show that compared with benchmark models, the SSGCN had an improved accuracy of 6.33–0.5%. In macro F1 evaluation, its improvement range was 11.68–0.5%.

Due to the limitations of traditional retinal blood vessel segmentation algorithms in feature extraction, vessel breakage often occurs at the end. To address this issue, a retinal vessel segmentation algorithm based on a modified U-shaped network is proposed in this paper. This algorithm can extract multi-scale vascular features and perform segmentation in an end-to-end manner. First, in order to improve the low contrast of the original image, pre-processing methods are employed. Second, a multi-scale residual convolution module is employed to extract image features of different granularities, while residual learning improves feature utilization efficiency and reduces information loss. In addition, a selective kernel unit is incorporated into the skip connections to obtain multi-scale features with varying receptive field sizes achieved through soft attention. Subsequently, to further extract vascular features and improve processing speed, a residual attention module is constructed at the decoder stage. Finally, a weighted joint loss function is implemented to address the imbalance between positive and negative samples. The experimental results on the DRIVE, STARE, and CHASE_DB1 datasets demonstrate that MU-Net exhibits better sensitivity and a higher Matthew’s correlation coefficient (0.8197, 0.8051; STARE: 0.8264, 0.7987; CHASE_DB1: 0.8313, 0.7960) compared to several state-of-the-art methods.

Rural building automatic extraction technology is of great significance for rural planning and disaster assessment; however, existing methods face the dilemma of scarce sample data and large regional differences in rural buildings. To solve this problem, this study constructed an image dataset of typical Chinese rural buildings, including nine typical geographical regions, such as the Northeast and North China Plains. Additionally, an improved remote sensing image rural building extraction network called AGSC-Net was designed. Based on an encoder–decoder structure, the model integrates multiple attention gate (AG) modules and a context collaboration network (CC-Net). The AG modules realize focused expression of building-related features through feature selection. The CC-Net module models the global dependency between different building instances, providing complementary localization and scale information to the decoder. By embedding AG and CC-Net modules between the encoder and decoder, the model can capture multiscale semantic information on building features. Experiments show that, compared with other models, AGSC-Net achieved the best quantitative metrics on two rural building datasets, verifying the accuracy of the extraction results. This study provides an effective example for automatic extraction in complex rural scenes and lays the foundation for related monitoring and planning applications.

The images captured using UAVs during inspection often contain a great deal of small targets related to transmission lines. These vulnerable elements are critical for ensuring the safe operation of these lines. However, due to various factors such as the small size of the targets, low resolution, complex background, and potential target aggregation, achieving accurate and real-time detection becomes challenging. To address these issues, this paper proposes a detection algorithm called P2-ECA-EIOU-YOLOv5 (P2E-YOLOv5). Firstly, to tackle the challenges posed by the issues of complex background and environmental interference impacting small targets, an ECA attention module is integrated into the network. The module effectively enhances the network’s focus on small targets, while concurrently mitigating the influence of environmental interference. Secondly, considering the characteristics of small target size and low resolution, a new high-resolution detection head is introduced, making the network more sensitive to small targets. Lastly, the network utilizes the EIOU_Loss as the regression loss function to improve the positioning accuracy of small targets, especially when they tend to aggregate. Experimental results demonstrate that the proposed P2E-YOLOv5 detection algorithm achieves an accuracy P (precision) of 96.0% and an average accuracy (mAP) of 97.0% for small-target detection in transmission lines.

Small objects detection is a challenging task in computer vision due to the limited semantic information that can be extracted and the susceptibility to background interference. In this paper, we propose a decoupled multi-scale small object detection algorithm named DMS-YOLOv5. The algorithm incorporates a receptive field module into the feature extraction network for better focus on low-resolution small objects. The coordinate attention mechanism, which combines spatial and channel attention information, is introduced to reduce interference from background information and enhance the network’s attention to object information. A detection layer tailored to small-sized objects is added to compensate for the loss of small object information in multiple downsampling operations, greatly improving the detection capability of small objects. Next, The decoupled network is introduced into the detection head network for branch processing of classification and bounding box regression tasks. Finally, the bounding box loss function is improved to alleviate missed detection problems caused by the concentration of small objects and mutual occlusion between objects. The improved method achieved a mean average precision improvement of 12.1% on VisDrone2019-DET dataset compared to the original method. In comparison experiments with similar methods, our proposed method also demonstrated good performance, validating its effectiveness.

In this work, a general theoretical framework for extractive summarization is proposed—the Attentional Extractive Summarization framework. Although abstractive approaches are generally used in text summarization today, extractive methods can be especially suitable for some applications, and they can help with other tasks such as Text Classification, Question Answering, and Information Extraction. The proposed approach is based on the interpretation of the attention mechanisms of hierarchical neural networks, which compute document-level representations of documents and summaries from sentence-level representations, which, in turn, are computed from word-level representations. The models proposed under this framework are able to automatically learn relationships among document and summary sentences, without requiring Oracle systems to compute the reference labels for each sentence before the training phase. These relationships are obtained as a result of a binary classification process, the goal of which is to distinguish correct summaries for documents. Two different systems, formalized under the proposed framework, were evaluated on the CNN/DailyMail and the NewsRoom corpora, which are some of the reference corpora in the most relevant works on text summarization. The results obtained during the evaluation support the adequacy of our proposal and suggest that there is still room for the improvement of our attentional framework.

Pests and diseases are an inevitable problem in agricultural production, causing substantial economic losses yearly. The application of convolutional neural networks to the intelligent recognition of crop pest images has become increasingly popular due to advances in deep learning methods and the rise of large-scale datasets. However, the diversity and complexity of pest samples, the size of sample images, and the number of examples all directly affect the performance of convolutional neural networks. Therefore, we designed a new target-detection framework based on Cascade RCNN (Regions with CNN features), aiming to solve the problems of large image size, many pest types, and small and unbalanced numbers of samples in pest sample datasets. Specifically, this study performed data enhancement on the original samples to solve the problem of a small and unbalanced number of examples in the dataset and developed a sliding window cropping method, which could increase the perceptual field to learn sample features more accurately and in more detail without changing the original image size. Secondly, combining the attention mechanism with the FPN (Feature Pyramid Networks) layer enabled the model to learn sample features that were more important for the current task from both channel and space aspects. Compared with the current popular target-detection frameworks, the average precision value of our model (mAP@0.5) was 84.16%, the value of (mAP@0.5:0.95) was 65.23%, the precision was 67.79%, and the F1 score was 82.34%. The experiments showed that our model solved the problem of convolutional neural networks being challenging to use because of the wide variety of pest types, the large size of sample images, and the difficulty of identifying tiny pests.

Breast cancer is one of the most common types of cancer among women. Accurate diagnosis at an early stage can reduce the mortality associated with this disease. Governments and health organizations stress the importance of early detection of breast cancer as it is related to an increase in the number of available treatment options and increased survival. Early detection gives patients the best chance of receiving effective treatment. Different types of images and imaging modalities are used in the detection and diagnosis of breast cancer. One of the imaging types is “infrared thermal” breast imaging, where a screening instrument is used to measure the temperature distribution of breast tissue. Although it has not been used often, compared to mammograms, it showed promising results when used for early detection. It also has many advantages as it is non-invasive, safe, painless, and inexpensive. The literature has indicated that the use of thermal images with deep neural networks improves the accuracy of early diagnosis of breast malformation. Therefore, in this paper, we aim to investigate to what extent convolutional neural networks (CNNs) with attention mechanisms (AMs) can provide satisfactory detection results in thermal breast cancer images. We present a model for breast cancer detection based on deep neural networks with AMs using thermal images from the Database for Research Mastology with Infrared Image (DMR-IR). The model will be evaluated in terms of accuracy, sensitivity and specificity, and will be compared against state-of-the-art breast cancer detection methods. The AMs with the CNN model achieved encouraging test accuracy rates of 99.46%, 99.37%, and 99.30% on the breast thermal dataset. The test accuracy of CNNs without AMs was 92.32%, whereas CNNs with AMs achieved an improvement in accuracy of 7%. Moreover, the proposed models outperformed previous models that were reviewed in the literature.

Strip steel is an important raw material for the related industries, such as aerospace, shipbuilding, and pipelines, and any quality defects in the strip steel would lead to huge economic losses. However, it is still a challenge task to effectively detect the defects from the background of the strip steel due to its complex variations, including variable flaws, chaotic background, and noise invasion. This paper proposes a novel strip steel defect detection method based on a U-shaped residual network, including an encoder and a decoder. The encoder is a fully convolutional neural network in which attention mechanisms are embedded to adequately extract multi-scale defect features and ro ignore irrelevant background regions. The decoder is a U-shaped residual network to capture more contextual data from different scales, without significantly increasing the computational cost due to the pooling operations used in the U-shaped network. Furthermore, a residual refinement module is designed immediately after the decoder to further optimize the coarse defect map. Experimental results show that the proposed method can effectively segment surface defect objects from irrelevant background noise and is superior to other advanced methods with clear boundaries.

How to achieve high-precision detection and real-time deployment of the lithology intelligent identification system has significant engineering implications in the geotechnical, geological, water conservation, and mining disciplines. In this study, a lightweight lithology intelligent identification model is proposed to overcome this problem. The MobileNetV2 model is utilized as the basic backbone network to decrease network operation parameters. Furthermore, channel attention and spatial attention methods are incorporated into the model to improve the network’s extraction of complicated and abstract petrographic elements. In addition, based on the findings of network training, computing power performance, test results, and Grad-CAM interpretability analysis and comparison tests with Resnet101, InceptionV3, and MobileNetV2 models. The training accuracy of the proposed model is 98.59 percent, the training duration is 76 min, and the trained model is just 6.38 megabytes in size. The precision (P), recall (R), and harmonic mean (FI-score) were, respectively, 89.62%, 91.38%, and 90.42%. Compared to the three competing models, the model presented in this work strikes a better balance between lithology recognition accuracy and speed, and it gives greater consideration to the rock feature area. Wider and more uniform, strong anti-interference capability, improved robustness and generalization performance of the model, which can be deployed in real-time on the client or edge devices and has some promotion value.

A visual dialog task entails an agent engaging in a multiple round conversation about an image. Notably, one of the main issues is capturing the semantic associations of multiple inputs, such as the questions, dialog history, and image features. Many of the techniques use a token or a sentence granularity semantic representation of the question and dialog history to model semantic associations; however, they do not perform collaborative modeling, which limits their efficacy. To overcome this limitation, we propose a multi-granularity semantic collaborative reasoning network to properly support a visual dialog. It employs different granularity semantic representations of the question and dialog history to collaboratively identify the relevant information from multiple inputs based on attention mechanisms. Specifically, the proposed method collaboratively reasons the question-related information from the dialog history based on its granular semantic representations. Then, it collaboratively locates the question-related visual objects in the image by leveraging refined question representations. The experimental results conducted on the VisDial v.1.0 dataset verify the effectiveness of the proposed method, showing the improvements of the best normalized discounted cumulative gain score from 59.37 to 60.98 with a single model, from 60.92 to 62.25 with ensemble models, and from 63.15 to 64.13 with performing multitask learning.

Automatic object detection by satellite remote sensing images is of great significance for resource exploration and natural disaster assessment. To solve existing problems in remote sensing image detection, this article proposes an improved YOLOX model for satellite remote sensing image automatic detection. This model is named RS-YOLOX. To strengthen the feature learning ability of the network, we used Efficient Channel Attention (ECA) in the backbone network of YOLOX and combined the Adaptively Spatial Feature Fusion (ASFF) with the neck network of YOLOX. To balance the numbers of positive and negative samples in training, we used the Varifocal Loss function. Finally, to obtain a high-performance remote sensing object detector, we combined the trained model with an open-source framework called Slicing Aided Hyper Inference (SAHI). This work evaluated models on three aerial remote sensing datasets (DOTA-v1.5, TGRS-HRRSD, and RSOD). Our comparative experiments demonstrate that our model has the highest accuracy in detecting objects in remote sensing image datasets.

The main objective of this paper is to use deep neural networks to decode the electroencephalography (EEG) signals evoked when individuals perceive four types of motion stimuli (contraction, expansion, rotation, and translation). Methods for single-trial and multi-trial EEG classification are both investigated in this study. Attention mechanisms and a variant of recurrent neural networks (RNNs) are incorporated as the decoding model. Attention mechanisms emphasize task-related responses and reduce redundant information of EEG, whereas RNN learns feature representations for classification from the processed EEG data. To promote generalization of the decoding model, a novel online data augmentation method that randomly averages EEG sequences to generate artificial signals is proposed for single-trial EEG. For our dataset, the data augmentation method improves the accuracy of our model (based on RNN) and two benchmark models (based on convolutional neural networks) by 5.60%, 3.92%, and 3.02%, respectively. The attention-based RNN reaches mean accuracies of 67.18% for single-trial EEG decoding with data augmentation. When performing multi-trial EEG classification, the amount of training data decreases linearly after averaging, which may result in poor generalization. To address this deficiency, we devised three schemes to randomly combine data for network training. Accordingly, the results indicate that the proposed strategies effectively prevent overfitting and improve the correct classification rate compared with averaging EEG fixedly (by up to 19.20%). The highest accuracy of the three strategies for multi-trial EEG classification achieves 82.92%. The decoding performance for the methods proposed in this work indicates they have application potential in the brain–computer interface (BCI) system based on visual motion perception.