Search

The joint utilization of multi-source data is of great significance in geospatial observation applications, such as urban planning, disaster assessment, and military applications. However, this approach is confronted with challenges including inconsistent data structures, irrelevant physical properties, scarce training data, insufficient utilization of information and an imperfect feature fusion method. Therefore, this paper proposes a novel binary-tree Transformer network (BTRF-Net), which is used to fuse heterogeneous information and utilize complementarity among multi-source remote sensing data to enhance the joint classification performance of hyperspectral image (HSI) and light detection and ranging (LiDAR) data. Firstly, a hyperspectral network (HSI-Net) is employed to extract spectral and spatial features of hyperspectral images, while the elevation information of LiDAR data is extracted using the LiDAR network (LiDAR-Net). Secondly, a multi-source transformer complementor (MSTC) is designed that utilizes the complementarity and cooperation among multi-modal feature information in remote sensing images to better capture their correlation. The multi-head complementarity attention mechanism (MHCA) within this complementor can effectively capture global features and local texture information of images, hence achieving full feature fusion. Then, to fully obtain feature information of multi-source remote sensing images, this paper designs a complete binary tree structure, binary feature search tree (BFST), which fuses multi-modal features at different network levels to obtain multiple image features with stronger representation abilities, effectively enhancing the stability and robustness of the network. Finally, several groups of experiments are designed to compare and analyze the proposed BTRF-Net with traditional methods and several advanced deep learning networks using two datasets: Houston and Trento. The results show that the proposed network outperforms other state-of-the-art methods even with small training samples. [ABSTRACT FROM AUTHOR]

Traffic accidents directly influence public safety and economic development; thus, the prevention of traffic accidents is of great importance in urban transportation. The accurate prediction of traffic accidents can assist traffic departments to better control and prevent accidents. Thus, this paper proposes a deep learning method named attention-based residual dilated network (ARDN), to extract essential information from multi-source datasets and enhance accident prediction accuracy. The method utilizes bidirectional long short-term memory to model sequential information and incorporates an attention mechanism to recalibrate weights. Furthermore, a dilated residual layer is adopted to capture long term information effectively. Feature encoding is also employed to incorporate natural language descriptions and point-of-interest data. Experimental evaluations of datasets collected from Austin and Houston demonstrate that ARDN outperforms a range of machine learning methods, such as logistic regression, gradient boosting, Xgboost, and deep learning methods. The ablation experiments further confirm the indispensability of each component in the proposed method. [ABSTRACT FROM AUTHOR]

Deep learning has achieved promising results for hyperspectral image (HSI) classification in recent years due to its hierarchical structure and automatic feature extraction ability from raw data. The HSI has continuous spectral information, allowing for the precise identification of materials by capturing minute spectral differences. Convolutional neural networks (CNNs) have proven to be effective feature extractors for HSI classification. However, inherent network limitations prevent them from adequately mining and representing the sequence attributes of spectral signatures and learning critical and valuable features from both spectral and spatial dimensions simultaneously. This paper proposes a deep learning-based framework called a novel dual attention-based multiscale-multilevel ConvLSTM3D (DAMCL) to address these challenges. In this work, our contribution is threefold; firstly, a dual attention mechanism is proposed, effectively learning critical and valuable features from spectral and spatial dimensions. Secondly, multiscale ConvLSTM3D blocks can learn the discriminative features alongside handling long-range dependencies of spectral data. Thirdly, these features are combined by a multilevel feature fusion approach to maximize the impact of features learned at different levels. To assess the performance of the proposed method, extensive experiments are carried out on five different benchmark datasets containing complex and challenging land cover classes. The results confirm that the proposed method outperforms state-of-the-art techniques with a small number of training samples in terms of overall accuracy (OA), average accuracy (AA), and Kappa (k). The overall accuracy of 98.88%, 99.42%, 99.20%, 95.37%, and 92.57% is achieved over the Indian Pines, Salinas Valley, University of Pavia, Houston 2013, and Houston 2018 datasets, respectively. [ABSTRACT FROM AUTHOR]

Deep learning has achieved impressive success in computer vision, especially remote sensing. It is well known that different deep models are able to extract different kinds of features from remote sensing images. For example, the convolutional neural networks (CNN) can extract neighbourhood spatial features in the short-range region, the graph convolutional networks (GCN) can extract structural features in the middle- and long-range region, and the encoder-decoder (ED) can obtain the reconstruction features from an image. Thus, it is challenging to design a model that can combine the different models to extract fused features in a hyperspectral image classification task. To this end, this paper proposes a three-branch attention deep model (TADM) for the classification of hyperspectral images. The model can be divided into three branches: graph convolutional neural network, convolutional neural network, and deep encoder-decoder. These three branches first extract structural features, spatial-spectral joint features and reconstructed encoded features from hyperspectral images, respectively. Then, a cross-fusion strategy and an attention mechanism are employed to automatically learn the fusion parameters and complete the feature fusion. Finally, the hybrid features are fed into a standard classifier for pixel-level classification. Extensive experiments on two real-world hyperspectral datasets (Houston and Trento) demonstrate the effectiveness and superiority of the proposed method. Compared with other baseline classification methods, such as FuNet-C and Two-Branch CNN(H), proposed method achieves the highest classification results. Specifically, overall classification accuracies of 93.25% and 95.84% were obtained on the Houston and Trento data, respectively. [ABSTRACT FROM AUTHOR]

Hyperspectral images contain rich features in both spectral and spatial domains, which bring opportunities for accurate recognition of similar materials and promote various fine-grained remote sensing applications. Although deep learning models have been extensively investigated in the field of hyperspectral image classification (HSIC) tasks, classification performance is still limited under small sample conditions, and this has been a longstanding problem. The features extracted by complex network structures with large model size are redundant to some extent and prone to overfitting. This paper proposes a low-rank constrained attention-enhanced multiple feature fusion network (LAMFN). Firstly, factor analysis is used to extract very few components that can describe the original data using covariance information to perform spectral feature preprocessing. Then, a lightweight attention-enhanced 3D convolution module is used for deep feature extraction, and the position-sensitive information is supplemented using a 2D coordinate attention. The above widely varying spatial–spectral feature groups are fused through a simple composite residual structure. Finally, low-rank second-order pooling is adopted to enhance the convolutional feature selectivity and achieve classification. Extensive experiments were conducted on four representative hyperspectral datasets with different spatial–spectral characteristics, namely Indian Pines (IP), Pavia Center (PC), Houston (HU), and WHU-HongHu (WHU). The contrast methods include several advanced models proposed recently, including residual CNNs, attention-based CNNs, and transformer-based models. Using only five samples per class for training, LAMFN achieved overall accuracies of 78.15%, 97.18%, 81.35%, and 87.93% on the above datasets, which has an improvement of 0.82%, 1.12%, 1.67%, and 0.89% compared to the second-best model. The running time of LAMFN is moderate. For example, the training time of LAMFN on the WHU dataset was 29.1 s, and the contrast models ranged from 3.0 s to 341.4 s. In addition, ablation experiments and comparisons with some advanced semi-supervised learning methods further validated the effectiveness of the proposed model designs. [ABSTRACT FROM AUTHOR]

Electricity price forecasts have become a fundamental factor affecting the decision-making of all market participants. Extreme price volatility has forced market participants to hedge against volume risks and price movements. Hence, getting an accurate price forecast from a few hours to a few days ahead is very important and very challenging due to various factors. This paper proposes an integrated long-term recurrent convolutional network (ILRCN) model to predict electricity prices considering the majority of contributing attributes to the market price as input. The proposed ILRCN model combines the functionalities of a convolutional neural network and long short-term memory (LSTM) algorithm along with the proposed novel conditional error correction term. The combined ILRCN model can identify the linear and nonlinear behavior within the input data. ERCOT wholesale market price data along with load profile, temperature, and other factors for the Houston region have been used to illustrate the proposed model. The performance of the proposed ILRCN electricity price forecasting model is verified using performance/evaluation metrics like mean absolute error and accuracy. Case studies reveal that the proposed ILRCN model shows the highest accuracy and efficiency in electricity price forecasting as compared to the support vector machine (SVM) model, fully connected neural network model, LSTM model, and the traditional LRCN model without the conditional error correction stage. [ABSTRACT FROM AUTHOR]

Extracting precise location information from microblogs is a crucial task in many applications, particularly in disaster response, revealing where damages are, where people need assistance, and where help can be found. A crucial prerequisite to location extraction is place name extraction. In this paper, we present GazPNE: a hybrid approach to place name extraction which fuses rules, gazetteers, and deep learning techniques without requiring any manually annotated data. The core of the approach is to learn the intrinsic characteristics of multi-word place names with deep learning from gazetteers. Specifically, GazPNE consists of a rule-based system to select n-grams from the microblogs that potentially contain place names, and a C-LSTM model that decides if the selected n-gram is a place name or not. The C-LSTM is trained on 388.1 million examples containing 6.8 million positive examples with US and Indian place names extracted from OpenStreetMap and 381.3 million negative examples synthesized by rules. We evaluate GazPNE against the SoTA on a manually annotated 4,500 tweet dataset which contains 9,026 place names from three foods: 2016 in Louisiana (US), 2016 in Houston (US), and 2015 in Chennai (India). GazPNE achieves SotA performance on the test data with an F1 of 0.84. [ABSTRACT FROM AUTHOR]

Multi-sensor remote sensing image classification has been considerably improved by deep learning feature extraction and classification networks. In this paper, we propose a novel multi-sensor fusion framework for the fusion of diverse remote sensing data sources. The novelty of this paper is grounded in three important design innovations: 1- a unique adaptation of the coupled residual networks to address multi-sensor data classification; 2- a smart auxiliary training via adjusting the loss function to address classifications with limited samples; and 3- a unique design of the residual blocks to reduce the computational complexity while preserving the discriminative characteristics of multi-sensor features. The proposed classification framework is evaluated using three different remote sensing datasets: the urban Houston university datasets (including Houston 2013 and the training portion of Houston 2018) and the rural Trento dataset. The proposed framework achieves high overall accuracies of 93.57%, 81.20%, and 98.81% on Houston 2013, the training portion of Houston 2018, and Trento datasets, respectively. Additionally, the experimental results demonstrate considerable improvements in classification accuracies compared with the existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In recent years, supervised learning-based methods have achieved excellent performance for hyperspectral image (HSI) classification. However, the collection of training samples with labels is not only costly but also time-consuming. This fact usually causes the existence of weak supervision, including incorrect supervision where mislabeled samples exist and incomplete supervision where unlabeled samples exist. Focusing on the inaccurate supervision and incomplete supervision, the weakly supervised classification of HSI is investigated in this paper. For inaccurate supervision, complementary learning (CL) is firstly introduced for HSI classification. Then, a new method, which is based on selective CL and convolutional neural network (SeCL-CNN), is proposed for classification with noisy labels. For incomplete supervision, a data augmentation-based method, which combines mixup and Pseudo-Label (Mix-PL) is proposed. And then, a classification method, which combines Mix-PL and CL (Mix-PL-CL), is designed aiming at better semi-supervised classification capacity of HSI. The proposed weakly supervised methods are evaluated on three widely-used hyperspectral datasets (i.e., Indian Pines, Houston, and Salinas datasets). The obtained results reveal that the proposed methods provide competitive results compared to the state-of-the-art methods. For inaccurate supervision, the proposed SeCL-CNN has outperformed the state-of-the-art method (i.e., SSDP-CNN) by 0.92%, 1.84%, and 1.75% in terms of OA on the three datasets, when the noise ratio is 30%. And for incomplete supervision, the proposed Mix-PL-CL has outperformed the state-of-the-art method (i.e., AROC-DP) by 1.03%, 0.70%, and 0.82% in terms of OA on the three datasets, with 25 training samples per class. [ABSTRACT FROM AUTHOR]