Search

To address the problems of noise interference and image blurring in hyperspectral imaging (HSI), this paper proposes a denoising method for HSI based on deep learning and a total variation (TV) prior. The method minimizes the first-order moment distance between the deep prior of a Fast and Flexible Denoising Convolutional Neural Network (FFDNet) and the Enhanced 3D TV (E3DTV) prior, obtaining dual priors that complement and reinforce each other’s advantages. Specifically, the original HSI is initially processed with a random binary sparse observation matrix to achieve a sparse representation. Subsequently, the plug-and-play (PnP) algorithm is employed within the framework of generalized alternating projection (GAP) to denoise the sparsely represented HSI. Experimental results demonstrate that, compared to existing methods, this method shows significant advantages in both quantitative and qualitative assessments, effectively enhancing the quality of HSIs.

UAV transmission tower inspection is the use of UAV technology for regular inspection and troubleshooting of towers on transmission lines, which helps to improve the safety and reliability of transmission lines and ensures the stability of the power supply. From the traditional manual tower boarding to the current way of manually selecting target camera shooting points from 3D point clouds to plan the inspection path of the UAV, operational efficiency has drastically improved. However, indoor planning work is still labor-consuming and expensive. In this paper, a deep learning-based point cloud transmission tower segmentation (PCTTS) model combined with the corresponding target point localization algorithm is proposed for automatic segmentation of transmission tower point cloud data and automatically localizing the key inspection component as the target point for UAV inspection. First, we utilize octree sampling with unit ball normalization to simplify the data and ensure translation invariance before putting the data into the model. In the feature extraction stage, we encode the point set information and combine Euclidean distance and cosine similarity features to ensure rotational invariance. On this basis, we adopt multi-scale feature extraction, construct a local coordinate system, and introduce the offset-attention mechanism to enhance model performance further. Then, after the feature propagation module, gradual up-sampling is used to obtain the features of each point to complete the point cloud segmentation. Finally, combining the segmentation results with the target point localization algorithm completes the automatic extraction of UAV inspection target points. The method has been applied to six kinds of transmission tower point cloud data of part segmentation results and three kinds of transmission tower point cloud data of instance segmentation results. The experimental results show that the model achieves mIOU of 94.1% on the self-built part segmentation dataset and 86.9% on the self-built instance segmentation dataset, and the segmentation accuracy outperforms that of the methods for point cloud segmentation, such as PointNet++, DGCNN, Point Transformer, and PointMLP. Meanwhile, the experimental results of UAV inspection target point localization also verify the method’s effectiveness in this paper.

Tree species identification is a critical component of forest resource monitoring, and timely and accurate acquisition of tree species information is the basis for sustainable forest management and resource assessment. Airborne hyperspectral images have rich spectral and spatial information and can detect subtle differences among tree species. To fully utilize the advantages of hyperspectral images, we propose a double-branch spatial–spectral joint network based on the SimAM attention mechanism for tree species classification. This method achieved high classification accuracy on three tree species datasets (93.31% OA value obtained in the TEF dataset, 95.7% in the Tiegang Reservoir dataset, and 98.82% in the Xiongan New Area dataset). The network consists of three parts: spectral branch, spatial branch, and feature fusion, and both branches make full use of the spatial–spectral information of pixels to avoid the loss of information. In addition, the SimAM attention mechanism is added to the feature fusion part of the network to refine the features to extract more critical features for high-precision tree species classification. To validate the robustness of the proposed method, we compared this method with other advanced classification methods through a series of experiments. The results show that: (1) Compared with traditional machine learning methods (SVM, RF) and other state-of-the-art deep learning methods, the proposed method achieved the highest classification accuracy in all three tree datasets. (2) Combining spatial and spectral information and incorporating the SimAM attention mechanism into the network can improve the classification accuracy of tree species, and the classification performance of the double-branch network is better than that of the single-branch network. (3) The proposed method obtains the highest accuracy under different training sample proportions, and does not change significantly with different training sample proportions, which are stable. This study demonstrates that high-precision tree species classification can be achieved using airborne hyperspectral images and the methods proposed in this study, which have great potential in investigating and monitoring forest resources.

Two space-borne light detection and ranging (LiDAR) missions, Global Ecosystem Dynamics Investigation (GEDI) and Ice, Cloud, and land Elevation Satellite-2 (ICESat-2), have demonstrated high capabilities in extracting terrain and canopy heights in forest environments. However, there have been limited studies evaluating their performance for terrain and canopy height retrievals in short-stature vegetation. This study utilizes airborne LiDAR data to validate and compare the accuracies of terrain and canopy height retrievals for short-stature vegetation using the latest versions of ICESat-2 (Version 5) and GEDI (Version 2). Furthermore, this study also analyzes the influence of various factors, such as vegetation type, terrain slope, canopy height, and canopy cover, on terrain and canopy height retrievals. The results indicate that ICESat-2 (bias = −0.05 m, RMSE = 0.67 m) outperforms GEDI (bias = 0.39 m, RMSE = 1.40 m) in terrain height extraction, with similar results observed for canopy height retrievals from both missions. Additionally, the findings reveal significant differences in terrain and canopy height retrieval accuracies between ICESat-2 and GEDI data under different data acquisition scenarios. Error analysis results demonstrate that terrain slope plays a pivotal role in influencing the accuracy of terrain height extraction for both missions, particularly for GEDI data, where the terrain height accuracy decreases significantly with increasing terrain slope. However, canopy height has the most substantial impact on the estimation accuracies of GEDI and ICESat-2 canopy heights. Overall, these findings confirm the strong potential of ICESat-2 data for terrain and canopy height retrievals in short-stature vegetation areas, and also provide valuable insights for future applications of space-borne LiDAR data in short-stature vegetation-dominated ecosystems.

With the continuous development of surface observation methods and technologies, we can acquire multiple sources of data more effectively in the same geographic area. The quality and availability of these data have also significantly improved. Consequently, how to better utilize multi-source data to represent ground information has become an important research question in the field of geoscience. In this paper, a novel model called multi-modal transformer cascaded fusion net (MMTCFN) is proposed for fusion and classification of multi-modal remote sensing data, Hyperspectral Imagery (HSI) and LiDAR data. Feature fusion and feature extraction are the two stages of the model. First, in the feature extraction stage, a three-branch cascaded Convolutional Neural Network (CNN) framework is employed to fully leverage the advantages of convolutional operators in extracting shallow-level local features. Based on this, we generated multi-modal long-range integrated deep features utilizing the transformer-based vectorized pixel group transformer (VPGT) module during the feature fusion stage. In the VPGT block, we designed a vectorized pixel group embedding that preserves the global features extracted from the three branches in a non-overlapping multi-space manner. Moreover, we introduce the DropKey mechanism into the multi-head self-attention (MHSA) to alleviate overfitting caused by insufficient training samples. Finally, we employ a probabilistic decision fusion strategy to integrate multiple class estimations, assigning a specific category to each pixel. This model was experimented on three HSI-LiDAR datasets with balanced and unbalanced training samples. The proposed model outperforms the other seven SOTA approaches in terms of OA performance, proving the superiority of MMTCFN for the HSI-LiDAR classification task.

Arc sag is an important parameter in the design and operation and maintenance of transmission lines and is directly related to the safety and reliability of grid operation. The current arc sag measurement method is inefficient and costly, which makes it difficult to meet the engineering demand for fast inspection of transmission lines. In view of this, this paper proposes an automatic spacer bar segmentation algorithm, CM-Mask-RCNN, that combines the CAB attention mechanism and MHSA self-attention mechanism, which automatically extracts the spacer bars and calculates the center coordinates, and combines classical algorithms such as beam method leveling, spatial front rendezvous, and spatial curve fitting, based on UAV inspection video data, to realize arc sag measurement with a low cost and high efficiency. It is experimentally verified that the CM-Mask-RCNN algorithm proposed in this paper achieves an AP index of 73.40% on the self-built dataset, which is better than the Yolact++, U-net, and Mask-RCNN algorithms. In addition, it is also verified that the adopted approach of fusing CAB and MHSA attention mechanisms can effectively improve the segmentation performance of the model, and this combination improves the model performance more significantly compared with other attention mechanisms, with an AP improvement of 2.24%. The algorithm in this paper was used to perform arc sag measurement experiments on 10 different transmission lines, and the measurement errors are all within ±2.5%, with an average error of −0.11, which verifies the effectiveness of the arc sag measurement method proposed in this paper for transmission lines.

We propose the Point Cloud Tree Species Classification Network (PCTSCN) to overcome challenges in classifying tree species from laser data with deep learning methods. The network is mainly composed of two parts: a sampling component in the early stage and a feature extraction component in the later stage. We used geometric sampling to extract regions with local features from the tree contours since these tend to be species-specific. Then we used an improved Farthest Point Sampling method to extract the features from a global perspective. We input the intensity of the tree point cloud as a dimensional feature and spatial information into the neural network and mapped it to higher dimensions for feature extraction. We used the data obtained by Terrestrial Laser Scanning (TLS) and Unmanned Aerial Vehicle Laser Scanning (UAVLS) to conduct tree species classification experiments of white birch and larch. The experimental results showed that in both the TLS and UAVLS datasets, the input tree point cloud density and the highest feature dimensionality of the mapping had an impact on the classification accuracy of the tree species. When the single tree sample obtained by TLS consisted of 1024 points and the highest dimension of the network mapping was 512, the classification accuracy of the trained model reached 96%. For the individual tree samples obtained by UAVLS, which consisted of 2048 points and had the highest dimension of the network mapping of 1024, the classification accuracy of the trained model reached 92%. TLS data tree species classification accuracy of PCTSCN was improved by 2–9% compared with other models using the same point density, amount of data and highest feature dimension. The classification accuracy of tree species obtained by UAVLS was up to 8% higher. We propose PCTSCN to provide a new strategy for the intelligent classification of forest tree species.

Although considerable success has been achieved in change detection on optical remote sensing images, accurate detection of specific changes is still challenging. Due to the diversity and complexity of the ground surface changes and the increasing demand for detecting changes that require high-level semantics, we have to resort to deep learning techniques to extract the intrinsic representations of changed areas. However, one key problem for developing deep learning metho for detecting specific change areas is the limitation of annotated data. In this paper, we collect a change detection dataset with 862 labeled image pairs, where the urban construction-related changes are labeled. Further, we propose a supervised change detection method based on a deep siamese semantic segmentation network to handle the proposed data effectively. The novelty of the method is that the proposed siamese network treats the change detection problem as a binary semantic segmentation task and learns to extract features from the image pairs directly. The siamese architecture as well as the elaborately designed semantic segmentation networks significantly improve the performance on change detection tasks. Experimental results demonstrate the promising performance of the proposed network compared to existing approaches.

3D building models are important for many applications related to human activities in urban environments. However, due to the high complexity of the building structures, it is still difficult to automatically reconstruct building models with accurate geometric description and semantic information. To simplify this problem, this article proposes a novel approach to automatically decompose the compound buildings with symmetric roofs into semantic primitives by exploiting local symmetry contained in the building structure. In this approach, the proposed decomposition allows the overlapping of neighbor primitives and each decomposed primitive can be represented as a parametric form, which simplify the complexity of the building reconstruction and facilitate the integration of LiDAR data and aerial imagery into a parameters optimization process. The proposed method starts by extracting isolated building regions from the LiDAR point clouds. Next, point clouds belonging to each compound building are segmented into planar patches to construct an attributed graph, and then the local symmetries contained in the attributed graph are exploited to automatically decompose the compound buildings into different semantic primitives. In the final step, 2D image features are extracted depending on the initial 3D primitives generated from LiDAR data, and then the compound building is reconstructed using constraints from LiDAR data and aerial imagery by a nonlinear least squares optimization. The proposed method is applied to two datasets with different point densities to show that the complexity of building reconstruction can be reduced considerably by decomposing the compound buildings into semantic primitives. The experimental results also demonstrate that the traditional model driven methods can be further extended to the automated reconstruction of compound buildings by using the proposed semantic decomposition method.

Monitoring variability in outlet glaciers can improve the understanding of feedbacks associated with calving, ocean thermal forcing, and climate change. In this study, we present a remote-sensing investigation of Dalk Glacier in East Antarctica to analyze its dynamic changes. Terminus positions and surface ice velocities were estimated from Landsat and Sentinel-1 data, and the high-precision Worldview digital elevation model (DEM) was generated to determine the location of the potential ice rumple. We detected the cyclic behavior of glacier terminus changes and similar periodic increases in surface velocity since 2000. The terminus retreated in 2006, 2009, 2010, and 2016 and advanced in other years. The surface velocity of Dalk Glacier has a 5-year cycle with interannual speed-ups in 2007, 2012, and 2017. Our observations show the relationship between velocity changes and terminus variations, as well as the driving role of the ice rumple. Ice velocity often increases after calving events and continuous retreats. The loss of buttressing provided by an ice rumple may be a primary factor for increases in ice velocity. Given the restriction of the ice rumple, the surface velocity remains relatively stable when the glacier advances. The calving events may be linked to the unstable terminus caused by the ice rumple.

In this paper, a primitive-based 3D building roof modeling method, by integrating LiDAR data and aerial imagery, is proposed. The novelty of the proposed modeling method is to represent building roofs by geometric primitives and to construct a cost function by using constraints from both LiDAR data and aerial imagery simultaneously, so that the accuracy potential of the different sensors can be tightly integrated for the building model generation by an integrated primitive’s parameter optimization procedure. To verify the proposed modeling method, both simulated data and real data with simple buildings provided by ISPRS (International Society for Photogrammetry and Remote Sensing), were used in this study. The experimental results were evaluated by the ISPRS, which demonstrate the proposed modeling method can integrate LiDAR data and aerial imagery to generate 3D building models with high accuracy in both the horizontal and vertical directions. The experimental results also show that by adding a component, such as a dormer, to the primitive, a variant of the simple primitive is constructed, and the proposed method can generate a building model with some details.

Tree stem detection is a key step toward retrieving detailed stem attributes from terrestrial laser scanning (TLS) data. Various point-based methods have been proposed for the stem point extraction at both individual tree and plot levels. The main limitation of the point-based methods is their high computing demand when dealing with plot-level TLS data. Although segment-based methods can reduce the computational burden and uncertainties of point cloud classification, its application is largely limited to urban scenes due to the complexity of the algorithm, as well as the conditions of natural forests. Here we propose a novel and simple segment-based method for efficient stem detection at the plot level, which is based on the curvature feature of the points and connected component segmentation. We tested our method using a public TLS dataset with six forest plots that were collected for the international TLS benchmarking project in Evo, Finland. Results showed that the mean accuracies of the stem point extraction were comparable to the state-of-art methods (>95%). The accuracies of the stem mappings were also comparable to the methods tested in the international TLS benchmarking project. Additionally, our method was applicable to a wide range of stem forms. In short, the proposed method is accurate and simple; it is a sensible solution for the stem detection of standing trees using TLS data.