Search

Simple Summary: Animal movement trajectories are effective indicators of key information such as social behavior, food acquisition, reproduction, migration, and survival strategies in animal behavior analysis. However, manual observation is still relied upon in many analysis scenarios, which is inefficient and error-prone. This paper introduces a computer vision-based method for tracking animal trajectories, which enables monitoring and accurate acquisition of individual target animal movement trajectories over extended periods, overcoming the limitations of manual observation. The experiments demonstrate that the method is efficient and accurate in tracking animals in complex scenes, providing essential basic data for animal behavior analysis and having a wide range of potential applications. Animal tracking is crucial for understanding migration, habitat selection, and behavior patterns. However, challenges in video data acquisition and the unpredictability of animal movements have hindered progress in this field. To address these challenges, we present a novel animal tracking method based on correlation filters. Our approach integrates hand-crafted features, deep features, and temporal context information to learn a rich feature representation of the target animal, enabling effective monitoring and updating of its state. Specifically, we extract hand-crafted histogram of oriented gradient features and deep features from different layers of the animal, creating tailored fusion features that encapsulate both appearance and motion characteristics. By analyzing the response map, we select optimal fusion features based on the oscillation degree. When the target animal's state changes significantly, we adaptively update the target model using temporal context information and robust feature data from the current frame. This updated model is then used for re-tracking, leading to improved results compared to recent mainstream algorithms, as demonstrated in extensive experiments conducted on our self-constructed animal datasets. By addressing specific challenges in animal tracking, our method offers a promising approach for more effective and accurate animal behavior research. [ABSTRACT FROM AUTHOR]

Deep learning has achieved remarkable success in the semantic segmentation of remote sensing images (RSIs).In the domain of semantic segmentation, where classification and localization tasks need to be performed simultaneously, it is crucial to consider both global and local spatial relationships in RSIs. This is especially important for the recognition of ground objects that have a slim and elongated appearance. However, existing methods for land use semantic segmentation lack an effective mechanism to coordinate and address these two aspects, resulting in limitations on the recognition of slim targets and the continuity of land object identification. Here, a specific attention-based network called PaANet is developed for semantic segmentation. Our proposed framework builds upon the Swin transformer by incorporating two key modules: the position-aware attention (PaA) module and the pyramid pooling expectation-maximization (PPEM) module. These modules provide significant improvements in recognition accuracy and the continuity of ground object recognition while preserving structural classification details. Furthermore, we propose a multiresolution data augmentation method that utilizes scale-related information to guide the encoder. This approach leads to improved performance and generalization ability for the model. In experiments, the mIoU of our approach for the BLU and GID datasets is 2.37% and 3.94% higher than that of the baseline model, respectively. Our results also show significant superior to those of other methods regarding the continuity of ground object recognition. [ABSTRACT FROM AUTHOR]

Remote sensing images (RSI) of urban regions often exhibit shadows cast by buildings and other objects, which may result in imprecise analysis and interpretation. Therefore, shadow detection plays a significant role in RSI scene understanding. Current approaches have paid little attention to the presence of pseudo shadows and their confounding effects on detection results. We tackle these questions in the spatial context of orientation-awareness, effectively capturing the intricate relationships between shadows and ground objects. Here, we introduce a novel deep learning network named the Orientation-Aware and Multi-Scale Feature Fusion Network (OAMSFNet) due to noise reduction considerations. The proposed OAMSFNet comprises Shadow Aware Feature Encoder (SFE), Orientation-Aware Context Module (OCM), and Multi-Scale Feature Pyramid (MFP). Furthermore, a multi-scale feature fusion algorithm has been devised to enhance the detection and segmentation capabilities of the model in shadow regions. Finally, a comparative study was conducted on the Aerial Imagery dataset for Shadow Detection (AISD) in both quantitative and qualitative aspects. The experimental results show that our method outperforms state-of-the-art methods while maintaining a lightweight model design, indicating that our approach exhibits remarkable accuracy and stability in excellent agreement with predictions. OAMSFNet achieved an average F-score of 89.85%, surpassing the straightforward semantic segmentation model SegNet by 9.94%, and demonstrated remarkable efficiency enhancements, with FLOPs reduced to as low as 11.48 G, resulting in efficiency improvements of 66.7%-96.5% compared to other shadow detection models. [ABSTRACT FROM AUTHOR]

Unmanned aerial vehicles (UAVs) open up new opportunities for transportation monitoring. However, the vehicle targets in UAV images are situated in dynamic scenarios, such as uncertain background, dramatically varying arrangement density, multi-scale, and arbitrary-oriented. Most strategies for UAV-based monitoring require complex manoeuvring and still lack accurate abilities and lightweight structures. Consequently, designing effective detection methods with both speed and accuracy is challenging. This paper proposes a lightweight YOLO-based arbitrary-oriented vehicle detector via precise positional information encoding and bidirectional feature fusion to address the above issues. First, an additional angular classification prediction branch is added to the YOLO head network to significantly improve the detection performance for arbitrary-oriented vehicles without incurring the extra computational complexity and burden. Second, a C3 module embedded coordinate attention (C3CA) is presented to capture long-range dependencies and preserve vehicles' precise positional information in feature maps. Then, a fully connected bidirectional feature fusion module (FC-BiFPN) is applied at the neck of the YOLO detection framework, which is helpful for multi-scale vehicle detection. This module can efficiently aggregate features at different resolutions and automatically enhance information interaction. Finally, experiments and comparisons on vehicle and remote sensing datasets demonstrate that our approach outperforms the state-of-the-art methods in balancing precision and efficiency. In addition, the overall network design follows the lightweight concept, which better meets the real-time requirements of the UAV urban traffic monitoring platform in realistic scenarios. [ABSTRACT FROM AUTHOR]

Remote sensing change detection (RSCD) is crucial for our understanding of the dynamic pattern of the Earth's surface and human influence. Recently, transformer-based methodologies have advanced from their powerful global modeling capabilities in RSCD tasks. Nevertheless, they remain under excessive parameterization, which continues to be severely constrained by time and computation resources. Here, we present a transformer-based RSCD model called the Segmentation Multi-Branch Change Detection Network (SMBCNet). Our proposed approach combines a hierarchically structured transformer encoder with a cross-scale enhancement module (CEM) to extract global information with lower complexity. To account for the diverse nature of changes, we introduce a plug-and-play multi-branch change fusion module (MCFM) that integrates temporal features. Within this module, we transform the change detection task into a semantic segmentation problem. Moreover, we identify the Temporal Feature Aggregation Module (TFAM) to facilitate integrating features from diverse spatial scales. These results demonstrate that semantic segmentation is an effective solution to change detection (CD) problems in remote sensing images. [ABSTRACT FROM AUTHOR]

Deep learning has proven to be highly successful at semantic segmentation of remote sensing images (RSIs); however, it remains challenging due to the significant intraclass variation and interclass similarity, which limit the accuracy and continuity of feature recognition in land use and land cover (LULC) applications. Here, we develop a stage-adaptive selective network that can significantly improve the accuracy and continuity of multiscale ground objects. Our proposed framework can learn to implement multiscale details based on a specific attention method (SaSPE) and transformer that work collectively. In addition, we enhance the feature extraction capability of the backbone network at both local and global scales by improving the window attention mechanism of the Swin Transfer. We experimentally demonstrate the success of this framework through quantitative and qualitative results. This study demonstrates the strong potential of the prior knowledge of deep learning-based models for semantic segmentation of RSIs. [ABSTRACT FROM AUTHOR]

Recent deep-learning successes have led to a new wave of semantic segmentation in remote sensing (RS) applications. However, most approaches rarely distinguish the role of the body and edge of RS ground objects; thus, our understanding of these semantic parts has been frustrated by the lack of detailed geometry and appearance. Here we present a multiscale decoupled supervision network for RS semantic segmentation. Our proposed framework extends a densely supervised encoder-decoder network with a feature decoupling module that can decouple semantic features with different scales into distinct body and edge components. We further conduct multiscale supervision of the original and decoupled body and edge features to enhance inner consistency and spatial boundaries in remote sensing image (RSI) ground objects, enabling new segmentation designs and semantic components that can learn to perform multiscale geometry and appearance. Our results outperform the previous algorithm and are robust to different datasets. These results demonstrate that decoupled supervision is an effective solution to semantic segmentation tasks of RS images. [ABSTRACT FROM AUTHOR]

Remote sensing image scene classification (RSISC), which aims to classify scene categories for remote sensing imagery, has broad applications in various fields. Recent deep learning (DL) successes have led to a new wave of RSISC applications; however, they lack explainability and trustworthiness. Here, we propose a bidirectional flow decision tree (BFDT) module to create a reliable RS scene classification framework. Our algorithm combines BFDT and Convolutional Neural Networks (CNNs) to make the decision process easily interpretable. First, we extract multilevel feature information from the pretrained CNN model, which provides the basis for constructing the subsequent hierarchical structure. Then the model uses the discriminative nature of scene features at different levels to gradually refine similar subsets and learn the interclass hierarchy. Meanwhile, the last fully connected layer embeds decision rules for the decision tree from the bottom up. Finally, the cascading softmax loss is used to train and learn the depth features based on the hierarchical structure formed by the tree structure that contains rich remote sensing information. We also discovered that superclass results can be obtained well for unseen classes due to its unique tree structure hierarchical property, which results in our model having a good generalization effect. The experimental results align with theoretical predictions using three popular datasets. Our proposed framework provides explainable results, leading to correctable and trustworthy approaches. [ABSTRACT FROM AUTHOR]

Simple Summary: Video recordings enable scientists to estimate species' presence, richness, abundance, demography, and activity. The increasing popularity of camera traps has led to a growing interest in developing approaches to more efficiently process images. Advanced artificial intelligence systems can automatically find and identify the species captured in the wild, but they are hampered by dependence on large samples. However, many species rarely occur, such as endangered species, and only a few shot samples are available. Building on recent advances in deep learning and few-shot learning technologies, we developed a multiobject-tracking approach based on a tracking-by-detection paradigm for wildlife to improve multiobject-tracking performance. We hope that it will be beneficial to ecology and wildlife biology by speeding up the process of multiobject tracking in the wild. Camera trapping and video recording are now ubiquitous in the study of animal ecology. These technologies hold great potential for wildlife tracking, but are limited by current learning approaches, and are hampered by dependence on large samples. Most species of wildlife are rarely captured by camera traps, and thus only a few shot samples are available for processing and subsequent identification. These drawbacks can be overcome in multiobject tracking by combining wildlife detection and tracking with few-shot learning. This work proposes a multiobject-tracking approach based on a tracking-by-detection paradigm for wildlife to improve detection and tracking performance. We used few-shot object detection to localize objects using a camera trap and direct video recordings that could augment the synthetically generated parts of separate images with spatial constraints. In addition, we introduced a trajectory reconstruction module for better association. It could alleviate a few-shot object detector's missed and false detections; in addition, it could optimize the target identification between consecutive frames. Our approach produced a fully automated pipeline for detecting and tracking wildlife from video records. The experimental results aligned with theoretical anticipation according to various evaluation metrics, and revealed the future potential of camera traps to address wildlife detection and tracking in behavior and conservation. [ABSTRACT FROM AUTHOR]

Image recording is now ubiquitous in the fields of endangered-animal conservation and GIS. However, endangered animals are rarely seen, and, thus, only a few samples of images of them are available. In particular, the study of endangered-animal detection has a vital spatial component. We propose an adaptive, few-shot learning approach to endangered-animal detection through data augmentation by applying constraints on the mixture of foreground and background images based on species distributions. First, the pre-trained, salient network U2-Net segments the foregrounds and backgrounds of images of endangered animals. Then, the pre-trained image completion network CR-Fill is used to repair the incomplete environment. Furthermore, our approach identifies a foreground–background mixture of different images to produce multiple new image examples, using the relation network to permit a more realistic mixture of foreground and background images. It does not require further supervision, and it is easy to embed into existing networks, which learn to compensate for the uncertainties and nonstationarities of few-shot learning. Our experimental results are in excellent agreement with theoretical predictions by different evaluation metrics, and they unveil the future potential of video surveillance to address endangered-animal detection in studies of their behavior and conservation. [ABSTRACT FROM AUTHOR]

Crowd anomaly detection is a practical and challenging problem to computer vision and VideoGIS due to abnormal events' rare and diverse nature. Consequently, traditional methods rely on low-level reconstruction in a single image space, easily affected by unimportant pixels or sudden variations. In addition, real-time detection for crowd anomaly detection is challenging, and localization of anomalies requires other supervision. We present a new detection approach to learn spatiotemporal features with the spatial constraints of a still dynamic image. First, a lightweight spatiotemporal autoencoder has been proposed, capable of real-time image reconstruction. Second, we offer a dynamic network to obtain a compact representation of video frames in motion, reducing false-positive anomaly alerts by spatial constraints. In addition, we adopt the perturbation visual interpretation method for anomaly visualization and localization to improve the credibility of the results. In experiments, our results provide competitive performance across various scenarios. Besides, our approach can process 52.9–63.4 fps in anomaly detection, making it practical for crowd anomaly detection in video surveillance. [ABSTRACT FROM AUTHOR]

The growing interest in deep learning approaches to video surveillance raises concerns about the accuracy and efficiency of neural networks. However, fast and reliable detection of abnormal events is still a challenging work. Here, we introduce a two-stream approach that offers an autoencoder-based structure for fast and efficient detection to facilitate anomaly detection from surveillance video without labeled abnormal events. Furthermore, we present post hoc interpretability of feature map visualization to show the process of feature learning, revealing uncertain and ambiguous decision boundaries in the video sequence. Experimental results on Avenue, UCSD Ped2, and Subway datasets show that our method can detect abnormal events well and explain the internal logic of the model at the object level. [ABSTRACT FROM AUTHOR]

Tourist image retrieval has attracted increasing attention from researchers. Mainly, supervised deep hash methods have significantly boosted the retrieval performance, which takes hand-crafted features as inputs and maps the high-dimensional binary feature vector to reduce feature-searching complexity. However, their performance depends on the supervised labels, but few labeled temporal and discriminative information is available in tourist images. This paper proposes an improved deep hash to learn enhanced hash codes for tourist image retrieval. It jointly determines image representations and hash functions with deep neural networks and simultaneously enhances the discriminative capability of tourist image hash codes with refined semantics of the accompanying relationship. Furthermore, we have tuned the CNN to implement end-to-end training hash mapping, calculating the semantic distance between two samples of the obtained binary codes. Experiments on various datasets demonstrate the superiority of the proposed approach compared to state-of-the-art shallow and deep hashing techniques. [ABSTRACT FROM AUTHOR]

Fine-grained primate behavior recognition is crucial for conservation efforts and longitudinal research, but a lack of suitable datasets and complex network structures has hindered its progress. To address these challenges, we propose a novel approach that employs a region-focused deep convolutional neural network and a labeled fine-grained recognition dataset. Our model leverages a progressive attention training strategy that emphasizes discriminative region attention and encourages complementarity with multiple leading levels. Additionally, we incorporate a region-focused image generator that identifies informative local regions and provides a priori knowledge for the subsequent layer. Our experiments demonstrate that our model surpasses state-of-the-art performance on fine-grained primate behavior recognition benchmark datasets, even outperforming some models by leveraging auxiliary information. Our approach produces the automated fine-grained action recognition of primate behavior, making it a significant step for exploiting large datasets in ethology and conservation. This approach can be readily applied to a more comprehensive range of species and scenarios. • We propose a deep learning model for identifying primate behaviors from still images. • We have developed a new fine-grained primate behavior detection dataset. • Experimental results demonstrate the effectiveness of our model. [ABSTRACT FROM AUTHOR]

The performance of convolutional neural network- (CNN-) based object detection has achieved incredible success. Howbeit, existing CNN-based algorithms suffer from a problem that small-scale objects are difficult to detect because it may have lost its response when the feature map has reached a certain depth, and it is common that the scale of objects (such as cars, buses, and pedestrians) contained in traffic images and videos varies greatly. In this paper, we present a 32-layer multibranch convolutional neural network named MBNet for fast detecting objects in traffic scenes. Our model utilizes three detection branches, in which feature maps with a size of 16 × 16, 32 × 32, and 64 × 64 are used, respectively, to optimize the detection for large-, medium-, and small-scale objects. By means of a multitask loss function, our model can be trained end-to-end. The experimental results show that our model achieves state-of-the-art performance in terms of precision and recall rate, and the detection speed (up to 33 fps) is fast, which can meet the real-time requirements of industry. [ABSTRACT FROM AUTHOR]

Deep learning-based methods for building extraction from remote sensing images have been widely applied in fields such as land management and urban planning. However, extracting buildings from remote sensing images commonly faces challenges due to specific shooting angles. First, there exists a foreground–background imbalance issue, and the model excessively learns features unrelated to buildings, resulting in performance degradation and propagative interference. Second, buildings have complex boundary information, while conventional network architectures fail to capture fine boundaries. In this paper, we designed a multi-task U-shaped network (BFL-Net) to solve these problems. This network enhances the expression of the foreground and boundary features in the prediction results through foreground learning and boundary refinement, respectively. Specifically, the Foreground Mining Module (FMM) utilizes the relationship between buildings and multi-scale scene spaces to explicitly model, extract, and learn foreground features, which can enhance foreground and related contextual features. The Dense Dilated Convolutional Residual Block (DDCResBlock) and the Dual Gate Boundary Refinement Module (DGBRM) individually process the diverted regular stream and boundary stream. The former can effectively expand the receptive field, and the latter utilizes spatial and channel gates to activate boundary features in low-level feature maps, helping the network refine boundaries. The predictions of the network for the building, foreground, and boundary are respectively supervised by ground truth. The experimental results on the WHU Building Aerial Imagery and Massachusetts Buildings Datasets show that the IoU scores of BFL-Net are 91.37% and 74.50%, respectively, surpassing state-of-the-art models. [ABSTRACT FROM AUTHOR]

With the popular use of geotagging images, more and more research efforts have been placed on geographical scene classification. In geographical scene classification, valid spatial feature selection can significantly boost the final performance. Bag of visual words (BoVW) can do well in selecting feature in geographical scene classification; nevertheless, it works effectively only if the provided feature extractor is well-matched. In this paper, we use convolutional neural networks (CNNs) for optimizing proposed feature extractor, so that it can learn more suitable visual vocabularies from the geotagging images. Our approach achieves better performance than BoVW as a tool for geographical scene classification, respectively, in three datasets which contain a variety of scene categories. [ABSTRACT FROM AUTHOR]

While deep learning (DL) has been making enormous strides in lipreading areas, it is still massively underused in learning, understanding, and producing human language content. Current DL lipreading methods rely on single-channel processing and monolingual datasets, which have a limited ability to adapt to cross-language applications. Here, we propose a novel lipreading driven deep learning framework to create cross-language learning patterns. To evaluate the algorithm's cross-language learning ability, we present a dataset CELR-200 for both Chinese and English in lipreading, containing 200-word classes with more than 80,000 samples. We also propose two Spatio-Temporal Reconstructed 3D convolutional kernels to reconstruct the 3D convolutional Spatio-Temporal relations. By using two STR-3D convolutional kernels, we present two new lipreading models, Serial-STRNet and Parallel-STRNet. These improvements reduce the number of 3D convolutional kernel parameters and improve performance, showing good performance in CELR-200 with 65.68% and 66.35%, respectively. They outperform other lipreading models, achieving an absolute improvement of 2.56% over the state-of-the-art model. Our results identify targets for future investigations and demonstrate that STR-3D convolutional kernels can provide critical insights into lipreading tasks. • A novel cross-language lipreading dataset was produced. • The Spatio-Temporal relationship of the 3D convolution kernel was reconstructed. • Two front-end modules were described and combined. • The experimental results achieved an absolute improvement of 2.56% over the state-of-the-art model. [ABSTRACT FROM AUTHOR]

Visual question answering (VQA), along with multiple types of image and textual questions, makes it a challenging task to infer the correct answer. Consequently, traditional methods rely on relevant cross-modal objects and seldom leverage the cooperation of visual appearance and textual understanding. Here, we present LRBNet, a model-based approach to the problem by analyzing VQA as a division of labor strategy. Using a dictionary and pre-trained GloVe vectors, we embedded region captions and questions by the LSTM. Furthermore, we use graphs to model image region captions and features and then feed them into two GNN-based networks to capture the semantic and visual relations. Finally, we modulate the vertex features of multimodal graphs. Thus, the question embedding and vertex features are fed into the multi-level answer predictor to produce the results. We experimentally validate that LRBNet is an exciting framework for visual–textual understanding and a more challenging alternative to better VQA because it needs to understand the image to be successful. Our study provides complementary prediction through hierarchical representation within and beyond the interactive understanding of the textual sequence and the images, and the experimental results show that LRBNet outperforms other leading models in most cases. • We propose a novel VQA model that simulates the human brain's physiological structure and function. • We provide an Interactive module to imitate the information exchange process of the corpus callosum. • Further, we show an ensemble learning-based answering strategy for our proposed model. • Our model achieves state-of-the-art performance compared to other leading VQA models. [ABSTRACT FROM AUTHOR]

Context-aware user interface plays an important role in many human-computer Interaction tasks of location based services. Although spatial models for context-aware systems have been studied extensively, how to locate specific spatial information for users is still not well resolved, which is important in the mobile environment where location based services users are impeded by device limitations. Better context-aware human-computer interaction models of mobile location based services are needed not just to predict performance outcomes, such as whether people will be able to find the information needed to complete a human-computer interaction task, but to understand human processes that interact in spatial query, which will in turn inform the detailed design of better user interfaces in mobile location based services. In this study, a context-aware adaptive model for mobile location based services interface is proposed, which contains three major sections: purpose, adjustment, and adaptation. Based on this model we try to describe the process of user operation and interface adaptation clearly through the dynamic interaction between users and the interface. Then we show how the model applies users’ demands in a complicated environment and suggested the feasibility by the experimental results. [ABSTRACT FROM AUTHOR]

In the field of object detection, feature pyramid network (FPN) can effectively extract multi-scale information. However, the majority of FPN-based methods suffer from a semantic gap between features of various sizes before feature fusion, which can lead to feature maps with significant aliasing. In this paper, we present a novel multi-scale semantic enhancement feature pyramid network (MSE-FPN) which consists of three effective modules: semantic enhancement module, semantic injection module, and gated channel guidance module to alleviate these problems. Specifically, inspired by the strong ability of the self-attention mechanism to model context, we propose a semantic enhancement module to model global context to obtain the global semantic information before feature fusion. Then we propose the semantic injection module to divide and merge global semantic information into feature maps at various scales to narrow the semantic gap between features at different scales and efficiently utilize the semantic information of high-level features. Finally, to mitigate feature aliasing caused by feature fusion, the gated channel guidance module selectively outputs crucial features via a gating unit. By replacing FPN with MSE-FPN in Faster R-CNN, our models achieve 39.4 and 41.2 Average precision (AP) using ResNet50 and ResNet101 as the backbone network respectively. When using ResNet-101-64x4d as the backbone, MSE-FPN achieved up to 43.4 AP. Our results demonstrate that replacing FPN with MSE-FPN significantly enhances the detection performance of state-of-the-art FPN-based detectors. [ABSTRACT FROM AUTHOR]

Video sensor networks are formed by the joining of heterogeneous sensor nodes, which is frequently reported as video of communication functionally bound to geographical locations. Decomposition of georeferenced video stream presents the expression of video from spatial feature set. Although it has been studied extensively, spatial relations underlying the scenario are not well understood, which are important to understand the semantics of georeferenced video and behavior of elements. Here we propose a method of mapping georeferenced video sequences for geographical scenes and use contextual random graphs to investigate semantic knowledge of georeferenced video, leading to correlation analysis of the target motion elements in the georeferenced video stream. We have used the connections of motion elements, both the correlation and continuity, to present a dynamic structure in time series that reveals clues to the event development of the video stream. Furthermore, we have provided a method for the effective integration of semantic and campaign information. Ultimately, the experimental results show that the provided method offers a better description of georeferenced video elements that cannot be achieved with existing schemes. In addition, it offers a new way of thinking for the semantic description of the georeferenced video scenarios. [ABSTRACT FROM AUTHOR]

Motion information plays a vital role in video action recognition, serving as the fundamental building basis for the accurate interpretation of dynamic sequences. However, extracting accurate motion details remains a significant challenge for two-dimensional (2D) CNNs. To address this issue, we present an action recognition framework, named accurate motion understanding network (AMU-Net), designed to effectively perceive and aggregate valuable motion cues. Specifically, AMU-Net is a 2D CNN equipped with the proposed accurate motion perceptron (AMP) and action graph module (AGM). To capture finer local motion details, the AMP is introduced to handle motion noise in temporal differences. This module enables the extraction of critical local motion patterns from bidirectional temporal differences and enhances action-related features. Furthermore, to learn more precise global motion representations, the AGM is introduced to address the spatial sparsity of motion objects by detecting motion objects and selectively aggregating their features using a graph reasoning framework. Extensive experiments are conducted on three public benchmarks: ActivityNet-200, UCF-101, and Kinetics-400. Experimental results demonstrate that the proposed AMU-Net (based on ResNet-50) outperforms recent 2D CNN-based methods with a comparable computational overhead. In addition, the experimental results also show the effective transferability of the two modules to three popular lightweight convolutional architectures, emphasizing their versatility. [ABSTRACT FROM AUTHOR]

Accurate traffic flow prediction is valuable for satisfying citizens' travel needs and alleviating urban traffic pressure. However, it is highly challenging due to the complexity of the urban geospatial structure and the highly nonlinear temporal and spatial dependence on human mobility. Most existing works proposed to rely on strict periods (e.g. daily and weekly) and separate the extraction of temporal and spatial features. Besides, most Recurrent Neural Network (RNN)-based models either fail to capture variations of spatial–temporal features in adjacent timestamps or ignore details of closeness. In this paper, we propose a M ulti-attention based H ybrid-convolution S patial-temporal R ecurrent N etwork (MHSRN) for region-based traffic flow prediction. In MHSRN, we leverage a hybrid-convolution module to capture both shifting features and rich information at the nearest timestamps, and we apply the downsampling procedure to reduce the computation of RNN-based model. Furthermore, we propose to adopt a space-aware multi-attention module to re-perceive global and local spatial–temporal features. We conduct extensive experiments based on three real-world datasets. The results show that the MHSRN outperforms other challenging baselines by approximately 0.2–8.1% in mean absolute error on all datasets. On datasets other than TaxiBJ, the MHSRN reduces the root mean square error by at least 2.8% compared with the RNN-based model. [ABSTRACT FROM AUTHOR]

In recent years, scene semantic recognition has become the most exciting and fastest growing research topic. Lots of scene semantic analysis methods thus have been proposed for better scene content interpretation. By using latent Dirichlet allocation (LDA) to deduce the effective topic features, the accuracy of image semantic recognition has been significantly improved. Besides, the method of extracting deep features by layer-by-layer iterative computation using convolutional neural networks (CNNs) has achieved great success in image recognition. The paper proposes a method called DF-LDA, which is a hybrid supervised–unsupervised method combined CNNs with LDA to extract image topics. This method uses CNNs to explore visual features that are more suitable for scene images, and group the features of salient semantics into visual topics through topic models. In contrast to the LDA as a tool for simply extracting image semantics, our approach achieves better performance on three datasets that contain various scene categories. [ABSTRACT FROM AUTHOR]

Unsupervised representation learning plays an important role in remote sensing image applications. Generative adversarial network (GAN) is the most popular unsupervised learning method in recent years. However, due to poor data augmentation, many GAN-based methods are often difficult to carry out. In this paper, we propose an improved unsupervised representation learning model called multi-layer feature fusion Wasserstein GAN (MF-WGANs) which considers extracting the feature information for remote sensing scene classification from unlabelled samples. First, we introduced a multi-feature fusion layer behind the discriminator to extract the high-level and mid-level feature information. Second, we combined the loss of multi-feature fusion layer and WGAN-GP to generate more stable and high-quality remote sensing images with a resolution of 256 × 256. Finally, the multi-layer perceptron classifier (MLP-classifier) is used to classify the features extracted from the multi-feature fusion layer and evaluated with the UC Merced Land-Use, AID and NWPU-RESISC45 data sets. Experiments show that MF-WGANs has richer data augmentation and better classification performance than other unsupervised representation learning classification models (e.g., MARTA GANs). [ABSTRACT FROM AUTHOR]

The purpose of aspect-based sentiment analysis (ABSA) is to identify the sentiment polarity of a given aspect of a sentence. Recent investigations have revealed that incorporating syntactic structures derived from dependency-parsing trees into graph convolutional networks (GCNs) can yield excellent performance. However, these GCN-based methods excessively rely on the quality of the dependency-parsing tree, resulting possibly in suboptimal dependencies between words. Moreover, these GCN-based models fail to adapt properly to informal and complex comments without syntactic dependencies. To alleviate these deficiencies, we proposed a target-based GCN with semantic and syntactic information (TSGCN). In a TSGCN, a new target generation (NTG) module with a dependency attention mechanism is designed to generate a new target representation using explicit semantic information to replace a given aspect. Then, the syntactic structure is reconstructed based on the new target representation to capture the shortest distance between the given aspect and viewpoint words. Finally, the semantic structure generated by the self-attention mechanism was injected into the syntactic structure to complement the semantic dependencies between words. The experimental findings on five benchmark datasets indicated that the TSGCN outperformed the other baseline models. [ABSTRACT FROM AUTHOR]

The primary goal of aspect-based sentiment analysis is to identify sentiment polarity concerning the given aspect in a sentence. Recent investigations have demonstrated the superior performance of graph convolutional neural network (GCN) on dependency parsing tree. However, these GCN-based models fail to take the given aspect into account when calculating the hidden node representation vector, as well as lack exploration of contextual commonsense knowledge. On the contrary, the gating mechanism enables the interaction of the context and the given aspect to enhance the impact of the given aspect on the context. Nevertheless, such interactions are frequently inadequate resulting in insufficient extraction of sentiment information. This paper proposes a dual-gated graph convolutional network via contextual affective knowledge (DGGCN) to address these issues. The core idea is to incorporate GCN into the gating mechanism to enhance GCN to fully aggregate node information while strengthening the concentration on the given aspect. Simultaneously, the incorporation of contextual affective knowledge into graph networks can refine the perception of affective features. Experimental findings on five benchmark datasets reveal that our proposed DGGCN surpasses state-of-the-art methods. [ABSTRACT FROM AUTHOR]