Search

This document is a summary of a special issue on advanced machine learning and deep learning techniques for remote sensing. The issue includes 16 research papers that cover a range of topics, including hyperspectral image classification, moving point target detection, radar echo extrapolation, and remote sensing object detection. Each paper introduces a novel approach or model and provides extensive testing and evaluation to demonstrate its effectiveness. The insights shared in this special issue are expected to contribute to future advancements in artificial intelligence-based remote sensing research. [Extracted from the article]

Side-scan sonar plays a crucial role in underwater exploration, and the autonomous detection of side-scan sonar images is vital for detecting unknown underwater environments. However, due to the complexity of the underwater environment, the presence of a few highlighted areas on the targets, blurred feature details, and difficulty in collecting data from side-scan sonar, achieving high-precision autonomous target recognition in side-scan sonar images is challenging. This article addresses this problem by improving the You Only Look Once v7 (YOLOv7) model to achieve high-precision object detection in side-scan sonar images. Firstly, given that side-scan sonar images contain large areas of irrelevant information, this paper introduces the Swin-Transformer for dynamic attention and global modeling, which enhances the model's focus on the target regions. Secondly, the Convolutional Block Attention Module (CBAM) is utilized to further improve feature representation and enhance the neural network model's accuracy. Lastly, to address the uncertainty of geometric features in side-scan sonar target features, this paper innovatively incorporates a feature scaling factor into the YOLOv7 model. The experiment initially verified the necessity of attention mechanisms in the public dataset. Subsequently, experiments on our side-scan sonar (SSS) image dataset show that the improved YOLOv7 model has 87.9% and 49.23% in its average accuracy ( m A P 0.5 ) and ( m A P 0.5:0.95), respectively. These results are 9.28% and 8.41% higher than the YOLOv7 model. The improved YOLOv7 algorithm proposed in this paper has great potential for object detection and the recognition of side-scan sonar images. [ABSTRACT FROM AUTHOR]

Visual object tracking is a key technology that is used in unmanned aerial vehicles (UAVs) to achieve autonomous navigation. In recent years, with the rapid development of deep learning, tracking algorithms based on Siamese neural networks have received widespread attention. However, because of complex and diverse tracking scenarios, as well as limited computational resources, most existing tracking algorithms struggle to ensure real-time stable operation while improving tracking performance. Therefore, studying efficient and fast-tracking frameworks, and enhancing the ability of algorithms to respond to complex scenarios has become crucial. Therefore, this paper proposes a cross-parallel attention and efficient match transformer for aerial tracking (SiamEMT). Firstly, we carefully designed the cross-parallel attention mechanism to encode global feature information and to achieve cross-dimensional interaction and feature correlation aggregation via parallel branches, highlighting feature saliency and reducing global redundancy information, as well as improving the tracking algorithm's ability to distinguish between targets and backgrounds. Meanwhile, we implemented an efficient match transformer to achieve feature matching. This network utilizes parallel, lightweight, multi-head attention mechanisms to pass template information to the search region features, better matching the global similarity between the template and search regions, and improving the algorithm's ability to perceive target location and feature information. Experiments on multiple drone public benchmark tests verified the accuracy and robustness of the proposed tracker in drone tracking scenarios. In addition, on the embedded artificial intelligence (AI) platform AGX Xavier, our algorithm achieved real-time tracking speed, indicating that our algorithm can be effectively applied to UAV tracking scenarios. [ABSTRACT FROM AUTHOR]

Contactless and non-destructive measuring tools can facilitate the moisture monitoring of bagged or bulk grain during transportation and storage. However, accurate target recognition and size prediction always impede the effectiveness of contactless monitoring in actual use. This paper developed a novel 3D reconstruction method upon multi-angle point clouds using a binocular depth camera and a proper Yolo-based neural model to resolve the problem. With this method, this paper developed an embedded and low-cost monitoring system for the in-warehouse grain bags, which predicted targets' 3D size and boosted contactless grain moisture measuring. Identifying and extracting the object of interest from the complex background was challenging in size prediction of the grain silo-bag on a conveyor. This study first evaluated a series of Yolo-based neural network models and explored the most appropriate neural network structure for accurately extracting the grain bag. In point-cloud processing, this study constructed a rotation matrix to fuse multi-angle point clouds to generate a complete one. This study deployed all the above methods on a Raspberry Pi-embedded board to perform the grain bag's 3D reconstruction and size prediction. For experimental validation, this study built the 3D reconstruction platform and tested grain bags' reconstruction performance. First, this study determined the appropriate positions (−60°, 0°, 60°) with the least positions and high reconstruction quality. Then, this study validated the efficacy of the embedded system by evaluating its speed and accuracy and comparing it to the original Torch model. Results demonstrated that the NCNN-accelerated model significantly enhanced the average processing speed, nearly 30 times faster than the Torch model. The proposed system predicted the objects' length, width, and height, achieving accuracies of 97.76%, 97.02%, and 96.81%, respectively. The maximum residual value was less than 9 mm. And all the root mean square errors were less than 7 mm. In the future, the system will mount three depth cameras for achieving real-time size prediction and introduce a contactless measuring tool to finalize grain moisture detection. [ABSTRACT FROM AUTHOR]

Deep learning has gained widespread interest in the task of building semantic segmentation modelling using remote sensing images; however, neural network models require a large number of training samples to achieve better classification performance, and the models are more sensitive to error patches in the training samples. The training samples obtained in semi-supervised classification methods need less reliable weakly labelled samples, but current semi-supervised classification research puts the generated weak samples directly into the model for applications, with less consideration of the impact of the accuracy and quality improvement of the weak samples on the subsequent model classification. Therefore, to address the problem of generating and optimising the quality of weak samples from training data in deep learning, this paper proposes a semi-supervised building classification framework. Firstly, based on the test results of the remote sensing image segmentation model and the unsupervised classification results of LiDAR point cloud data, this paper quickly generates weak image samples of buildings. Secondly, in order to improve the quality of the spots of the weak samples, an iterative optimisation strategy of the weak samples is proposed to compare and analyse the weak samples with the real samples and extract the accurate samples from the weak samples. Finally, the real samples, the weak samples, and the optimised weak samples are input into the semantic segmentation model of buildings for accuracy evaluation and analysis. The effectiveness of this paper's approach was experimentally verified on two different building datasets, and the optimised weak samples improved by 1.9% and 0.6%, respectively, in the test accuracy mIoU compared to the initial weak samples. The results demonstrate that the semi-supervised classification framework proposed in this paper can be used to alleviate the model's demand for a large number of real-labelled samples while improving the ability to utilise weak samples, and it can be used as an alternative to fully supervised classification methods in deep learning model applications that require a large number of training samples. [ABSTRACT FROM AUTHOR]

Generative adversarial networks (or GANs) are a specific deep learning architecture often used for different usages, such as data generation or image-to-image translation. In recent years, this structure has gained increased popularity and has been used in different fields. One area of expertise currently in vogue is the use of GANs to produce synthetic aperture radar (SAR) data, and especially expand training datasets for SAR automatic target recognition (ATR). In effect, the complex SAR image formation makes these kind of data rich in information, leading to the use of deep networks in deep learning-based methods. Yet, deep networks also require sufficient data for training. However, contrary to optical images, we generally do not have a substantial number of available SAR images because of their acquisition and labelling cost; GANs are then an interesting tool. Concurrently, how to improve explainability for SAR ATR deep neural networks and how to make their reasoning more transparent have been increasingly explored as model opacity deteriorates trust of users. This paper aims at reviewing how GANs are used with SAR images, but also giving perspectives on how GANs could be used to improve interpretability and explainability of SAR classifiers. [ABSTRACT FROM AUTHOR]

Floods are among the most common disasters, causing loss of life and enormous damage to private property and public infrastructure. Monitoring systems that detect and predict floods help respond quickly in the pre-disaster phase to prevent and mitigate flood risk and damages. Thus, this paper presents a deep neural network (DNN)-based real-time flood monitoring system for embedded Internet of Things (IoT) edge devices. The proposed system fuses long-wave infrared (LWIR) and RGB cameras to overcome a critical drawback of conventional RGB camera-based systems: severe performance deterioration at night. This system recognizes areas occupied by water using a DNN-based semantic segmentation network, whose input is a combination of RGB and LWIR images. Flood warning levels are predicted based on the water occupancy ratio calculated by the water segmentation result. The warning information is delivered to authorized personnel via a mobile message service. For real-time edge computing, the heavy semantic segmentation network is simplified by removing unimportant channels while maintaining performance by utilizing the network slimming technique. Experiments were conducted based on the dataset acquired from the sensor module with RGB and LWIR cameras installed in a flood-prone area. The results revealed that the proposed system successfully conducts water segmentation and correctly sends flood warning messages in both daytime and nighttime. Furthermore, all of the algorithms in this system were embedded on an embedded IoT edge device with a Qualcomm QCS610 System on Chip (SoC) and operated in real time. [ABSTRACT FROM AUTHOR]

Forest biomass estimation is undoubtedly one of the most pressing research subjects at present. Combining multi-source remote sensing information can give full play to the advantages of different remote sensing technologies, providing more comprehensive and rich information for aboveground biomass (AGB) estimation research. Based on Landsat 8, Sentinel-2A, and ALOS2 PALSAR data, this paper takes the artificial coniferous forests in the Saihanba Forest of Hebei Province as the object of study, fully explores and establishes remote sensing factors and information related to forest structure, gives full play to the advantages of spectral signals in detecting the horizontal structure and multi-dimensional synthetic aperture radar (SAR) data in detecting the vertical structure, and combines environmental factors to carry out multivariate synergistic methods of estimating the AGB. This paper uses three variable selection methods (Pearson correlation coefficient, random forest significance, and the least absolute shrinkage and selection operator (LASSO)) to establish the variable sets, combining them with three typical non-parametric models to estimate AGB, namely, random forest (RF), support vector regression (SVR), and artificial neural network (ANN), to analyze the effect of forest structure on biomass estimation, explore the suitable AGB of artificial coniferous forests estimation of machine learning models, and develop the method of quantifying saturation value of the combined variables. The results show that the horizontal structure is more capable of explaining the AGB compared to the vertical structure information, and that combining the multi-structure information can improve the model results and the saturation value to a great extent. In this study, different sets of variables can produce relatively superior results in different models. The variable set selected using LASSO gives the best results in the SVR model, with an R 2 values of 0.9998 and 0.8792 for the training and the test set, respectively, and the highest saturation value obtained is 185.73 t/ha, which is beyond the range of the measured data. The problem of saturation in biomass estimation in boreal medium- and high-density forests was overcome to a certain extent, and the AGB of the Saihanba area was better estimated. [ABSTRACT FROM AUTHOR]

The accurate prediction of heavy precipitation in convective environments is crucial because such events, often occurring in Italy during the summer and fall seasons, can be a threat for people and properties. In this paper, we analyse the impact of satellite-derived surface-rainfall-rate data assimilation on the Weather Research and Forecasting (WRF) model's precipitation prediction, considering 15 days in summer 2022 and 17 days in fall 2022, where moderate to intense precipitation was observed over Italy. A 3DVar realised at CNR-ISAC (National Research Council of Italy, Institute of Atmospheric Sciences and Climate) is used to assimilate two different satellite-derived rain rate products, both exploiting geostationary (GEO), infrared (IR), and low-Earth-orbit (LEO) microwave (MW) measurements: One is based on an artificial neural network (NN), and the other one is the operational P-IN-SEVIRI-PMW product (H60), delivered in near-real time by the EUMETSAT HSAF (Satellite Application Facility in Support of Operational Hydrology and Water Management). The forecast is verified in two periods: the hours from 1 to 4 (1–4 h phase) and the hours from 3 to 6 (3–6 h phase) after the assimilation. The results show that the rain rate assimilation improves the precipitation forecast in both seasons and for both forecast phases, even if the improvement in the 3–6 h phase is found mainly in summer. The assimilation of H60 produces a high number of false alarms, which has a negative impact on the forecast, especially for intense events (30 mm/3 h). The assimilation of the NN rain rate gives more balanced predictions, improving the control forecast without significantly increasing false alarms. [ABSTRACT FROM AUTHOR]

Accurately classifying the intra-pulse modulations of radar emitter signals is important for radar systems and can provide necessary information for relevant military command strategy and decision making. As strong additional white Gaussian noise (AWGN) leads to a lower signal-to-noise ratio (SNR) of received signals, which results in a poor classification accuracy on the classification models based on deep neural networks (DNNs), in this paper, we propose an effective denoising method based on a denoising diffusion probabilistic model (DDPM) for increasing the quality of signals. Trained with denoised signals, classification models can classify samples denoised by our method with better accuracy. The experiments based on three DNN classification models using different modal input, with undenoised data, data denoised by the convolutional denoising auto-encoder (CDAE), and our method's denoised data, are conducted with three different conditions. The extensive experimental results indicate that our proposed method could denoise samples with lower values of the SNR, and that it is more effective for increasing the accuracy of DNN classification models for radar emitter signal intra-pulse modulations, where the average accuracy is increased from around 3 to 22 percentage points based on three different conditions. [ABSTRACT FROM AUTHOR]

In remote sensing scene classification (RSSC), restrictions on real-time processing on power consumption, performance, and resources necessitate the compression of neural networks. Unlike artificial neural networks (ANNs), spiking neural networks (SNNs) convey information through spikes, offering superior energy efficiency and biological plausibility. However, the high latency of SNNs restricts their practical application in RSSC. Therefore, there is an urgent need to research ultra-low-latency SNNs. As latency decreases, the performance of the SNN significantly deteriorates. To address this challenge, we propose a novel spatio-temporal pruning method that enhances the feature capture capability of ultra-low-latency SNNs. Our approach integrates spatial fundamental structures during the training process, which are subsequently pruned. We conduct a comprehensive evaluation of the impacts of these structures across classic network architectures, such as VGG and ResNet, demonstrating the generalizability of our method. Furthermore, we develop an ultra-low-latency training framework for SNNs to validate the effectiveness of our approach. In this paper, we successfully achieve high-performance ultra-low-latency SNNs with a single time step for the first time in RSSC. Remarkably, our SNN with one time step achieves at least 200 times faster inference time while maintaining a performance comparable to those of other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The integration of multi-source remote sensing data, bolstered by advancements in deep learning, has emerged as a pivotal strategy for enhancing land use and land cover (LULC) classification accuracy. However, current methods often fail to consider the numerous prior knowledge of remote sensing images and the characteristics of heterogeneous remote sensing data, resulting in data loss between different modalities and the loss of a significant amount of useful information, thus affecting classification accuracy. To tackle these challenges, this paper proposes a LULC classification method based on remote sensing data that combines a Transformer and cross-pseudo-siamese learning deep neural network (TCPSNet). It first conducts shallow feature extraction in a dynamic multi-scale manner, fully leveraging the prior information of remote sensing data. Then, it further models deep features through the multimodal cross-attention module (MCAM) and cross-pseudo-siamese learning module (CPSLM). Finally, it achieves comprehensive fusion of local and global features through feature-level fusion and decision-level fusion combinations. Extensive experiments on datasets such as Trento, Houston 2013, Augsburg, MUUFL and Berlin demonstrate the superior performance of the proposed TCPSNet. The overall accuracy (OA) of the network on the Trento, Houston 2013 and Augsburg datasets is of 99.76%, 99.92%, 97.41%, 87.97% and 97.96%, respectively. [ABSTRACT FROM AUTHOR]

Deep learning has brought great development to radar emitter identification technology. In addition, specific emitter identification (SEI), as a branch of radar emitter identification, has also benefited from it. However, the complexity of most deep learning algorithms makes it difficult to adapt to the requirements of the low power consumption and high-performance processing of SEI on embedded devices, so this article proposes solutions from the aspects of software and hardware. From the software side, we design a Transformer variant network, lightweight convolutional Transformer (LW-CT) that supports parameter sharing. Then, we cascade convolutional neural networks (CNNs) and the LW-CT to construct a one-dimensional-CNN-Transformer(1D-CNN-Transformer) lightweight neural network model that can capture the long-range dependencies of radar emitter signals and extract signal spatial domain features meanwhile. In terms of hardware, we design a low-power neural network accelerator based on an FPGA to complete the real-time recognition of radar emitter signals. The accelerator not only designs high-efficiency computing engines for the network, but also devises a reconfigurable buffer called "Ping-pong CBUF" and two-level pipeline architecture for the convolution layer for alleviating the bottleneck caused by the off-chip storage access bandwidth. Experimental results show that the algorithm can achieve a high recognition performance of SEI with a low calculation overhead. In addition, the hardware acceleration platform not only perfectly meets the requirements of the radar emitter recognition system for low power consumption and high-performance processing, but also outperforms the accelerators in other papers in terms of the energy efficiency ratio of Transformer layer processing. [ABSTRACT FROM AUTHOR]

In radar High-Resolution Range Profile (HRRP) target recognition, the targets of interest are always non-cooperative, posing a significant challenge in acquiring sufficient samples. This limitation results in the prevalent issue of limited sample availability. To mitigate this problem, researchers have sought to integrate handcrafted features into deep neural networks, thereby augmenting the information content. Nevertheless, existing methodologies for fusing handcrafted and deep features often resort to simplistic addition or concatenation approaches, which fail to fully capitalize on the complementary strengths of both feature types. To address these shortcomings, this paper introduces a novel radar HRRP feature fusion technique grounded in the Feature Weight Assignment Generative Adversarial Network (FWA-GAN) framework. This method leverages the generative adversarial network architecture to facilitate feature fusion in an innovative manner. Specifically, it employs the Feature Weight Assignment Model (FWA) to adaptively assign attention weights to both handcrafted and deep features. This approach enables a more efficient utilization and seamless integration of both feature modalities, thereby enhancing the overall recognition performance under conditions of limited sample availability. As a result, the recognition rate increases by over 4% compared to other state-of-the-art methods on both the simulation and experimental datasets. [ABSTRACT FROM AUTHOR]

In recent years, the detection performance of SAR-GMTI (synthetic aperture radar-ground moving target indication) algorithm based on deep learning has always been limited by insufficient measured data due to the heavy operation complexity and high cost of real SAR systems. To solve this problem, this paper proposes an overall DT-based implementation framework for SAR ground moving target intelligent detection tasks. In particular, by virtue of a SAR imaging algorithm, a high-fidelity twin replica of SAR moving targets is established in digital space through parameter traversal based on the prior target characteristics of the obtained measured datasets. Then, the constructed SAR twin datasets is fed into the neural network model to train an intelligent detector by fully learning features of the moving targets and preset the SAR scene in the twin space, which can realize the robust detection of ground moving targets in related practical scenarios with no need for multiple and complex field experiments. Moreover, the effectiveness of the proposed framework is verified on the MiniSAR measured system, and a comparison with traditional CFAR detection method is given simultaneously. [ABSTRACT FROM AUTHOR]

As the pre-part of tasks such as fault detection and line inspection, insulator detection is a crucial task. However, considering the complex environment of high-voltage transmission lines, the traditional insulator detection accuracy is unsatisfactory, and manual inspection is dangerous and inefficient. To improve this situation, this paper proposes an insulator detection model Siamese ID-YOLO based on a deep neural network. The model achieves the best balance between speed and accuracy compared with traditional detection methods. In order to achieve the purpose of image enhancement, this paper adopts the canny-based edge detection operator to highlight the edges of insulators to obtain more semantic information. In this paper, based on the Darknet53 network and Siamese network, the insulator original image and the edge image are jointly input into the model. Siamese IN-YOLO model achieves more fine-grained extraction of insulators through weight sharing between Siamese networks, thereby improving the detection accuracy of insulators. This paper uses statistical clustering analysis on the area and aspect ratio of the insulator data set, then pre-set and adjusts the hyperparameters of the model anchor box to make it more suitable for the insulator detection task. In addition, this paper makes an insulator dataset named InsuDaSet based on UAV(Unmanned Aerial Vehicle) shoot insulator images for model training. The experiments show that the insulator detection can reach 92.72% detection accuracy and 84FPS detection speed, which can fully meet the online insulator detection requirements. [ABSTRACT FROM AUTHOR]

Deep neural networks have achieved remarkable results in the field of polarimetric synthetic aperture radar (PolSAR) image classification. However, PolSAR is affected by speckle imaging, resulting in PolSAR images usually containing a large amount of speckle noise, which usually leads to the poor spatial consistency of classification results and insufficient classification accuracy. Semantic segmentation methods based on deep learning can realize the task of image segmentation and classification at the same time, producing fine-grained and smooth classification maps. However, these approaches require enormous labeled data sets, which are laborious and time-consuming. Due to these issues, a new multi-modal contrastive fully convolutional network, named MCFCN, is proposed for PolSAR image classification in this paper, which combines multi-modal features of the same pixel as inputs to the model based on a fully convolutional network and accomplishes the classification task using only a small amount of labeled data through contrastive learning. In addition, to describe the PolSAR terrain targets more comprehensively and enhance the robustness of the classifier, a pixel overlapping classification strategy is proposed, which can not only improve the classification accuracy effectively but also enhance the stability of the method. The experiments demonstrate that compared with existing classification methods, the classification results of the proposed method for three real PolSAR datasets have higher classification accuracy. [ABSTRACT FROM AUTHOR]

Efficient and effective radar emitter recognition is critical for electronic support measurement (ESM) systems. However, in complex electromagnetic environments, intercepted pulse trains generally contain substantial data noise, including spurious and missing pulses. Currently, radar emitter recognition methods utilizing traditional artificial neural networks (ANNs) like CNNs and RNNs are susceptible to data noise and require intensive computations, posing challenges to meeting the performance demands of modern ESM systems. Spiking neural networks (SNNs) exhibit stronger representational capabilities compared to traditional ANNs due to the temporal dynamics of spiking neurons and richer information encoded in precise spike timing. Furthermore, SNNs achieve higher computational efficiency by performing event-driven sparse addition calculations. In this paper, a lightweight spiking neural network is proposed by combining direct coding, leaky integrate-and-fire (LIF) neurons, and surrogate gradients to recognize radar emitters. Additionally, an improved SNN for radar emitter recognition is proposed, leveraging the local timing structure of pulses to enhance adaptability to data noise. Simulation results demonstrate the superior performance of the proposed method over existing methods. [ABSTRACT FROM AUTHOR]

In this paper, a technology named SCM−ANN combining physical scattering mechanisms and artificial intelligence is proposed to realize radar cross-section (RCS) extrapolation of non-cooperative conductor targets with higher efficiency. Firstly, an adaptive scattering center (SC) extraction algorithm is used to construct the scattering center model (SCM) for non-cooperative targets from radar echoes in the low-frequency band (LFB). Secondly, an artificial neural network (ANN) is constructed to capture the nonlinear relationship between the real LFB echoes and those reconstructed from the SCM. Finally, the SCM is used to reconstruct echoes in the high-frequency band (HFB), and these reconstructions, together with the trained ANN, optimize the extrapolated HFB RCS. For the SCM−ANN technology, physical mechanistic modes are used for trend prediction, and artificial intelligence is used for regression optimization based on trend prediction. Simulation results show that the proposed method can achieve a 50% frequency extrapolation range, with an average prediction error reduction of up to 40% compared with the traditional scheme. By incorporating physical mechanisms, this proposed approach offers improved accuracy and an extended extrapolation range compared with the RCS extrapolation techniques relying solely on numerical prediction. [ABSTRACT FROM AUTHOR]

As the demand for spatial positioning continues to grow, positioning methods based on inertial measurement units (IMUs) are emerging as a promising research topic due to their low cost and robustness against environmental interference. These methods are particularly well suited for global navigation satellite system (GNSS)-denied environments and challenging visual scenarios. While existing algorithms for position estimation using IMUs have demonstrated some effectiveness, there is still significant room for improvement in terms of estimation accuracy. Current approaches primarily treat IMU data as simple time series, neglecting the frequency-domain characteristics of IMU signals. This paper emphasizes the importance of frequency-domain information in IMU signals and proposes a novel neural network, WINNet (Wavelet Inertial Neural Network), which integrates time- and frequency-domain signals using a wavelet transform for spatial positioning with inertial sensors. Additionally, we collected ground-truth data using a LiDAR setup and combined it with the TLIO dataset to form a new IMU spatial positioning dataset. The experimental results demonstrate that our proposed method outperforms the current state-of-the-art inertial neural network algorithms in terms of the ATE, RTE, and drift error metrics overall. [ABSTRACT FROM AUTHOR]

The anthropogenic climate crisis results in the gradual loss of tree species in locations where they were previously able to grow. This leads to increasing workloads and requirements for foresters and arborists as they are forced to restructure their forests and city parks. The advancements in computer vision (CV)—especially in supervised deep learning (DL)—can help cope with these new tasks. However, they rely on large, carefully annotated datasets to produce good and generalizable models. This paper presents BAMFORESTS: a dataset with 27,160 individually delineated tree crowns in 105 ha of very-high-resolution UAV imagery gathered with two different sensors from two drones. BAMFORESTS covers four areas of coniferous, mixed, and deciduous forests and city parks. The labels contain instance segmentations of individual trees, and the proposed splits are balanced by tree species and vitality. Furthermore, the dataset contains the corrected digital surface model (DSM), representing tree heights. BAMFORESTS is annotated in the COCO format and is especially suited for training deep neural networks (DNNs) to solve instance segmentation tasks. BAMFORESTS was created in the BaKIM project and is freely available under the CC BY 4.0 license. [ABSTRACT FROM AUTHOR]

The measurement of sea surface height (SSH), which is of great importance in the field of oceanography, can be obtained through the innovative technique of GNSS-R for remote sensing. This research utilizes the dataset from spaceborne GNSS-R platforms, the Cyclone Global Navigation Satellite System (CYGNSS) and FengYun-3E (FY-3E), as the primary source of data for retrieving sea surface height (SSH). The utilization of artificial neural networks (ANNs) allows for the accurate estimation of ocean surface height with a precision of meter-level accuracy throughout the period of 1–17 August 2022. As a traditional machine learning method, an ANN is employed to extract pertinent data features, facilitating the acquisition of precise sea surface height estimations. Additionally, separate models are devised for both GNSS-R platforms, one based on constant velocity (CV) and the other on constant acceleration (CA). The Interactive Multiple Model (IMM) is utilized as the main method to combine the four models and convert the likelihood of each model. The transition between the models allows the filters to effectively adapt to dynamic changes and complex environments. This approach relies on the fundamental notion of the Kalman filter (KF), which showcases robust noise handling capabilities in predicting the SSH, separately. The results demonstrate that the model interaction technology is capable of efficiently filtering and integrating SSH data, yielding a Root Mean Square Error (RMSE) of 1.03 m. This corresponds to a 9.84% enhancement compared to the retrieved height from CYGNSS and a 37.19% enhancement compared to the retrieved height from FY-3E. The model proposed in this paper provides a potential scheme for the GNSS-R data fusion of multiple platforms and multiple models. In the future, more data sources and more models can be added to achieve more accurate adaptive fusion. [ABSTRACT FROM AUTHOR]

In recent years, the use of deep neural network in effective network feature extraction and the design of efficient and high-precision hyperspectral image classification algorithms has gradually become a research hotspot for scholars. However, due to the difficulty of obtaining hyperspectral images and the high cost of annotation, the training samples are very limited. In order to cope with the small sample problem, researchers often deepen the network model and use the attention mechanism to extract features; however, as the network model continues to deepen, the gradient disappears, the feature extraction ability is insufficient, and the computational cost is high. Therefore, how to make full use of the spectral and spatial information in limited samples has gradually become a difficult problem. In order to cope with such problems, this paper proposes two-branch multiscale spatial–spectral feature aggregation with a self-attention mechanism for a hyperspectral image classification model (FHDANet); the model constructs a dense two-branch pyramid structure, which can achieve the high efficiency extraction of joint spatial–spectral feature information and spectral feature information, reduce feature loss to a large extent, and strengthen the model's ability to extract contextual information. A channel–space attention module, ECBAM, is proposed, which greatly improves the extraction ability of the model for salient features, and a spatial information extraction module based on the deep feature fusion strategy HLDFF is proposed, which fully strengthens feature reusability and mitigates the feature loss problem brought about by the deepening of the model. Compared with five hyperspectral image classification algorithms, SVM, SSRN, A2S2K-ResNet, HyBridSN, SSDGL, RSSGL and LANet, this method significantly improves the classification performance on four representative datasets. Experiments have demonstrated that FHDANet can better extract and utilise the spatial and spectral information in hyperspectral images with excellent classification performance under small sample conditions. [ABSTRACT FROM AUTHOR]

In this paper, we propose a method for efficient target classification based on the spatial features of the point cloud generated by using a high-resolution radar sensor. The frequency-modulated continuous wave radar sensor can estimate the distance and velocity of a target. In addition, the azimuth and elevation angle of the target can be estimated by using a multiple-input and multiple-output antenna system. Using the estimated distance, velocity, and angle, the 3D point cloud of target can be generated. From the generated point cloud, we extract the point cloud for each individual target using the density-based spatial clustering of application with noise method and a camera mounted on the radar sensor. Then, we define the convex hull boundaries that enclose these point clouds in both 3D and 2D spaces obtained by orthogonally projecting onto the x y , y z , and z x planes. Using the vertices of convex hull, we calculate the volume of the targets and the areas in 2D spaces. Several feature points, including the calculated spatial information, are numerized and configured into feature vectors. We design an uncomplicated deep neural network classifier based on minimal input information to achieve fast and efficient classification performance. As a result, the proposed method achieved an average accuracy of 97.1%, and the time required for training was reduced compared to the method using only point cloud data and the convolutional neural network-based method. [ABSTRACT FROM AUTHOR]

Water bodies can offer local microclimates that have the potential to attenuate the effects of urban heat islands by reducing local temperature. This capability is shaded when the river is channelized. In such cases, the river temperature rises during hot periods, leading to negative impacts on the water quality. The main aim of this paper is to develop a local simple model to predict the temperature of the Dâmbovița River at its exit from Bucharest City, the capital of Romania. The location is chosen based on the historical critical impacts, in terms of extreme heatwaves that took place during hot summers, as well as future possible risks due to climate change. The water temperature prediction model is based on an artificial neural network that uses the Levenberg–Marquardt algorithm, due to its stability and rapid convergence capabilities. The model forecasts, with an accuracy of ±1 °C, the water temperature in an ungauged, downstream location, as a function of measured air and upstream water temperatures. The proposed model represents a first attempt to provide water managers in Bucharest City with a useful tool that will allow them to take timely measures to counteract the unwanted effects that can be generated by high water temperatures. [ABSTRACT FROM AUTHOR]

Ship classification using the synthetic aperture radar (SAR) images has a significant role in remote sensing applications. Aiming at the problems of excessive model parameters numbers and high energy consumption in the traditional deep learning methods for the SAR ship classification, this paper provides an energy-efficient SAR ship classification paradigm that combines spiking neural networks (SNNs) with Siamese network architecture, for the first time in the field of SAR ship classification, which is called the Siam-SpikingShipCLSNet. It combines the advantage of SNNs in energy consumption and the advantage of the idea in performances that use the Siamese neuron network to fuse the features from dual-polarized SAR images. Additionally, we migrated the feature fusion strategy from CNN-based Siamese neural networks to the SNN domain and analyzed the effects of various spiking feature fusion methods on the Siamese SNN. Finally, an end-to-end error backpropagation optimization method based on the surrogate gradient has been adopted to train this model. Experimental results tested on the OpenSARShip2.0 dataset have demonstrated the correctness and effectiveness of the proposed SAR ship classification strategy, which has the advantages of the higher accuracy, fewer parameters and lower energy consumption compared with the mainstream deep learning method of the SAR ship classification. [ABSTRACT FROM AUTHOR]

Sea ice is an important component of the polar circle and influences atmospheric change. Global navigation satellite system reflectometry (GNSS-R) not only realizes time-continuous and wide-area sea ice detection, but also greatly reduces the cost of sea ice remote sensing research, which has been a hot topic in recent years. To tackle the challenges of noise interference and the reduced accuracy of sea ice detection during the melting period, this paper proposes a sea ice detection method based on a residual neural network (ResNet). ResNet addresses the issue of vanishing gradients in deep neural networks and introduces residual connections, which allows the network to reuse learned features from previous layers. Delay-Doppler maps (DDMs) collected from TechDemoSat-1 (TDS-1) are used as input, and National Oceanic and Atmospheric Administration (NOAA) surface-type data above 60°N are selected as the true values. Based on ResNet, the sea ice detection achieved an accuracy of 98.61%, demonstrating high robustness to noise and strong stability during the sea ice melting period (June to September). In comparison to other sea ice detection algorithms, it stands out with its advantages of high accuracy, stability, and insensitivity to noise. [ABSTRACT FROM AUTHOR]

Due to the inherent periodicity of the angle, the direction of the object detected by the current rotating object detection algorithm is fuzzy. In order to solve this problem, this paper proposes a rotating object direction estimation method based on a neural network, which determines the unique direction of the object by predicting the direction vector of the object. Firstly, we use the two components (sin θ , cos θ) of the direction vector and the length and width parameters of the object to express the object model. Secondly, we construct a neural network model to predict the parameters used to express the object model. However, there is a constraint that the sum of the squares of the two components of the direction vector of the object is equal to 1. Because each output element of the neural network is independent, it is difficult to learn the constrained data between such neurons. Therefore, the function transformation model is designed, and the network transformation layer is added. Finally, affine transformation is used to transform the object parameters and carry out regression calculation, so as to detect the object and determine the direction of the object at the same time. This paper uses three sets of data to carry out the experiment, which are DOTA 1.5, HRSC, and UCAS-AOD data sets. It can be seen from the experimental results that for the object with correct ground truth, the proposed method can not only locate the object but also estimate the direction of the object accurately. [ABSTRACT FROM AUTHOR]

In this paper, a lightweight deep neural network (DNN) for direction of arrival (DOA) estimation is proposed, of which the input vector is designed to remove data redundancy as well as remaining DOA information. By exploring the Vandermonde property of the steering vector of a uniform linear array (ULA), the size of the newly designed input vector is greatly reduced. Furthermore, the DOA estimation is designed as a regression problem instead of a classification problem; that is, the lightweight DNN designs the output vector as the estimated DOAs of sources, of which the size is much shorter than that of the spatial spectrum used as the output vector in the conventional DNN. The reductions in the sizes of input and output vectors lead to a reduction in the sizes of hidden layers, achieving lightweightness of the neural network. The analysis illustrates that when the number of sensors is 22, the number of parameters in the lightweight DNN is three orders of magnitude less than that in the conventional DNN. The simulation results demonstrate the lightweight DNN can provide high DOA estimation accuracy with the shortest testing time. It performs better than the conventional DNN. Furthermore, it is superior to traditional solutions such as the multiple signal classification (MUSIC) method and conventional beamforming (CBF) method in harsh conditions like low signal-to-noise ratios (SNRs), closely spaced sources, and few snapshots. [ABSTRACT FROM AUTHOR]

Oceanic targets, including ripples, islands, vessels, and coastlines, display distinct sparse characteristics, rendering the ocean a significant arena for sparse Synthetic Aperture Radar (SAR) imaging rooted in sparse signal processing. Deep neural networks (DNNs), a current research emphasis, have, when integrated with sparse SAR, attracted notable attention for their exceptional imaging capabilities and high computational efficiency. Yet, the efficiency of traditional unfolding techniques is impeded by their architecturally inefficient design, which curtails their information transmission capacity and consequently detracts from the quality of reconstruction. This paper unveils a novel Memory-Augmented Deep Unfolding Network (MADUN) for SAR imaging in marine environments. Our methodology harnesses the synergies between deep learning and algorithmic unfolding, enhanced with a memory component, to elevate SAR imaging's computational precision. At the heart of our investigation is the incorporation of High-Throughput Short-Term Memory (HSM) and Cross-Stage Long-Term Memory (CLM) within the MADUN framework, ensuring robust information flow across unfolding stages and solidifying the foundation for deep, long-term informational correlations. Our experimental results demonstrate that our strategy significantly surpasses existing methods in enhancing the reconstruction of sparse marine scenes. [ABSTRACT FROM AUTHOR]

In synthetic aperture radar (SAR) imaging, intelligent object detection methods are facing significant challenges in terms of model robustness and application security, which are posed by adversarial examples. The existing adversarial example generation methods for SAR object detection can be divided into two main types: global perturbation attacks and local perturbation attacks. Due to the dynamic changes and irregular spatial distribution of SAR coherent speckle backgrounds, the attack effectiveness of global perturbation attacks is significantly reduced by coherent speckle. In contrast, by focusing on the image objects, local perturbation attacks achieve targeted and effective advantages over global perturbations by minimizing interference from the SAR coherent speckle background. However, the adaptability of conventional local perturbations is limited because they employ a fixed size without considering the diverse sizes and shapes of SAR objects under various conditions. This paper presents a framework for region-adaptive local perturbations (RaLP) specifically designed for SAR object detection tasks. The framework consists of two modules. To address the issue of coherent speckle noise interference in SAR imagery, we develop a local perturbation generator (LPG) module. By filtering the original image, this module reduces the speckle features introduced during perturbation generation. It then superimposes adversarial perturbations in the form of local perturbations on areas of the object with weaker speckles, thereby reducing the mutual interference between coherent speckles and adversarial perturbation. To address the issue of insufficient adaptability in terms of the size variation in local adversarial perturbations, we propose an adaptive perturbation optimizer (APO) module. This optimizer adapts the size of the adversarial perturbations based on the size and shape of the object, effectively solving the problem of adaptive perturbation size and enhancing the universality of the attack. The experimental results show that RaLP reduces the detection accuracy of the YOLOv3 detector by 29.0%, 29.9%, and 32.3% on the SSDD, SAR-Ship, and AIR-SARShip datasets, respectively, and the model-to-model and dataset-to-dataset transferability of RaLP attacks are verified. [ABSTRACT FROM AUTHOR]

Hyperspectral remote sensing images (HSIs) have both spectral and spatial characteristics. The adept exploitation of these attributes is central to enhancing the classification accuracy of HSIs. In order to effectively utilize spatial and spectral features to classify HSIs, this paper proposes a method for the spatial feature extraction of HSIs based on a mutually guided image filter (muGIF) and combined with the band-distance-grouped principal component. Firstly, aiming at the problem that previously guided image filtering cannot effectively deal with the inconsistent information structure between the guided and target information, a method for extracting spatial features using muGIF is proposed. Then, aiming at the problem of the information loss caused by a single principal component as a guided image in the traditional GIF-based spatial–spectral classification, a spatial feature-extraction framework based on the band-distance-grouped principal component is proposed. The method groups the bands according to the band distance and extracts the principal components of each set of band subsets as the guide map of the current band subset to filter the HSIs. A deep convolutional neural network model and a generative adversarial network model for the filtered HSIs are constructed and then trained using samples for HSIs' spatial–spectral classification. Experiments show that compared with the traditional methods and several popular spatial–spectral HSI classification methods based on a filter, the proposed methods based on muGIF can effectively extract the spatial–spectral features and improve the classification accuracy of HSIs. [ABSTRACT FROM AUTHOR]

Optical remote sensing videos, as a new source of remote sensing data that has emerged in recent years, have significant potential in remote sensing applications, especially national defense. In this paper, a tracking pipeline named TDNet (tracking while detecting based on a neural network) is proposed for optical remote sensing videos based on a correlation filter and deep neural networks. The pipeline is used to simultaneously track ships and planes in videos. There are many target tracking methods for general video data, but they suffer some difficulties in remote sensing videos with low resolution and those influenced by weather conditions. The tracked targets are usually misty. Therefore, in TDNet, we propose a new multi-target tracking method called MT-KCF and a detecting-assisted tracking (i.e., DAT) module to improve tracking accuracy and precision. Meanwhile, we also design a new target recognition (i.e., NTR) module to recognise newly emerged targets. In order to verify the performance of TDNet, we compare our method with several state-of-the-art tracking methods on optical video remote sensing data sets acquired from the Jilin No. 1 satellite. The experimental results demonstrate the effectiveness and the state-of-the-art performance of the proposed method. The proposed method can achieve more than 90% performance in terms of precision for single-target tracking tasks and more than 85% performance in terms of MOTA for multi-object tracking tasks. [ABSTRACT FROM AUTHOR]

This paper presents a method for predicting atmospheric duct conditions from a clutter power spectrum using deep learning. To accurately predict the duct conditions, deep learning with a binary classification is applied to the proposed refractivity from the clutter (RFC) method. The input data set is the artificial clutter data that are generated via the Advanced Refractive Prediction System (AREPS) simulation software Ver. 3.6 in conjunction with random atmospheric refractive indices. The output of the RFC method is then predicted via binary classification, indicating whether the atmospheric conditions are duct or non-duct. For the cross-validation, the clutter power spectrum data are generated based on real atmospheric refractivity data. The results show that the DNN trained with 5600 pieces of data (validation accuracy of 95.99%) exhibits a binary classification accuracy of 98.36%. The deep neural network (DNN) trained with 28,000 pieces of data (validation accuracy of 98.20%) achieves a binary classification accuracy of 99.06% with an F1-score of 0.9921. [ABSTRACT FROM AUTHOR]

The accurate semantic segmentation of point cloud data is the basis for their application in the inspection of extra high-voltage transmission lines (EHVTL). As deep learning evolves, point-wise-based deep neural networks have shown great potential for the semantic segmentation of EHVTL point clouds. However, EHVTL point cloud data are characterized by a large data volume and significant class imbalance. Therefore, the down-sampling method and point cloud feature extraction method used in current point-wise-based deep neural networks hardly meet the needs of computational accuracy and efficiency. In this paper, we proposed a two-step down-sampling method and a point cloud feature extraction method based on local feature aggregation of the point clouds after down-sampling in each layer of the model (LFAPAD). We then established a deep neural network named PowerLine-Net for the semantic segmentation of the EHVTL point clouds. Furthermore, in order to test and analyze the performance of PowerLine-Net, we constructed a point cloud dataset for the EHVTL scenes. Using this dataset and the Semantic3D dataset, we implemented network parameter testing, semantic segmentation, and an accuracy comparison of different networks based on PowerLine-Net. The results illustrate that the semantic segmentation model proposed in this paper has a high computational efficiency and accuracy in the semantic segmentation of EHVTL point clouds. Compared with conventional deep neural networks, including PointCNN, KPConv, SPG, PointNet++, and RandLA-Net, PowerLine-Net also achieves a higher accuracy in the semantic segmentation of EHVTL point clouds. Moreover, based on the results predicted by PowerLine-Net, the risk point detection for EHVTL point clouds has been achieved, which demonstrates the important value of this network in practical applications. In addition, as shown by the results of Semantic3D, PowerLine-Net also achieves a high segmentation accuracy, which proves its powerful capability and wide applicability in semantic segmentation for the point clouds of large-scale scenes. [ABSTRACT FROM AUTHOR]

Remote sensing change detection involves detecting pixels that have changed from a bi-temporal image of the same location. Current mainstream change detection models use encoder-decoder structures as well as Siamese networks. However, there are still some challenges with this: (1) Existing change feature fusion approaches do not take into account the symmetry of change features, which leads to information loss; (2) The encoder is independent of the change detection task, and feature extraction is performed separately for dual-time images, which leads to underutilization of the encoder parameters; (3) There are problems of unbalanced positive and negative samples and bad edge region detection. To solve the above problems, a mutual feature-aware network (MFNet) is proposed in this paper. Three modules are proposed for the purpose: (1) A symmetric change feature fusion module (SCFM), which uses double-branch feature selection without losing feature information and focuses explicitly on focal spatial regions based on cosine similarity to introduce strong a priori information; (2) A mutual feature-aware module (MFAM), which introduces change features in advance at the encoder stage and uses a cross-type attention mechanism for long-range dependence modeling; (3) A loss function for edge regions. After detailed experiments, the F1 scores of MFNet on SYSU-CD and LEVIR-CD were 83.11% and 91.52%, respectively, outperforming several advanced algorithms, demonstrating the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Marine oil spills can cause serious damage to marine ecosystems and biological species, and the pollution is difficult to repair in the short term. Accurate oil type identification and oil thickness quantification are of great significance for marine oil spill emergency response and damage assessment. In recent years, hyperspectral remote sensing technology has become an effective means to monitor marine oil spills. The spectral and spatial features of oil spill images at different levels are different. To accurately identify oil spill types and quantify oil film thickness, and perform better extraction of spectral and spatial features, a multilevel spatial and spectral feature extraction network is proposed in this study. First, the graph convolutional neural network and graph attentional neural network models were used to extract spectral and spatial features in non-Euclidean space, respectively, and then the designed modules based on 2D expansion convolution, depth convolution, and point convolution were applied to extract feature information in Euclidean space; after that, a multilevel feature fusion method was developed to fuse the obtained spatial and spectral features in Euclidean space in a complementary way to obtain multilevel features. Finally, the multilevel features were fused at the feature level to obtain the oil spill information. The experimental results show that compared with CGCNN, SSRN, and A2S2KResNet algorithms, the accuracy of oil type identification and oil film thickness classification of the proposed method in this paper is improved by 12.82%, 0.06%, and 0.08% and 2.23%, 0.69%, and 0.47%, respectively, which proves that the method in this paper can effectively extract oil spill information and identify different oil spill types and different oil film thicknesses. [ABSTRACT FROM AUTHOR]

Due to the complex interaction of urban and mountainous floods, assessing flood susceptibility in mountainous urban areas presents a challenging task in environmental research and risk analysis. Data-driven machine learning methods can evaluate flood susceptibility in mountainous urban areas lacking essential hydrological data, utilizing remote sensing data and limited historical inundation records. In this study, two ensemble learning algorithms, Random Forest (RF) and XGBoost, were adopted to assess the flood susceptibility of Kunming, a typical mountainous urban area prone to severe flood disasters. A flood inventory was created using flood observations from 2018 to 2022. The spatial database included 10 explanatory factors, encompassing climatic, geomorphic, and anthropogenic factors. Artificial Neural Network (ANN) and Support Vector Machine (SVM) were selected for model comparison. To minimize the influence of expert opinions on model training, this study employed a strategy of uniformly random sampling in historically non-flooded areas for negative sample selection. The results demonstrated that (1) ensemble learning algorithms offer higher accuracy than other machine learning methods, with RF achieving the highest accuracy, evidenced by an area under the curve (AUC) of 0.87, followed by XGBoost at 0.84, surpassing both ANN (0.83) and SVM (0.82); (2) the interpretability of ensemble learning highlighted the differences in the potential distribution of the training data's positive and negative samples. Feature importance in ensemble learning can be utilized to minimize human bias in the collection of flooded-site samples, more targeted flood susceptibility maps of the study area's road network were obtained; and (3) ensemble learning algorithms exhibited greater stability and robustness in datasets with varied negative samples, as evidenced by their performance in F1-Score, Kappa, and AUC metrics. This paper further substantiates the superiority of ensemble learning in flood susceptibility assessment tasks from the perspectives of accuracy, interpretability, and robustness, enhances the understanding of the impact of negative samples on such assessments, and optimizes the specific process for urban flood susceptibility assessment using data-driven methods. [ABSTRACT FROM AUTHOR]

Cones are among the significant and controversial landforms on Mars. Martian cones exhibit various morphological characteristics owing to their complex origin, and their precise origin remains an active research topic. A limited number of cones have been manually mapped from high-resolution images in local areas, and existing detection methods are only applicable to a single type of cone that has a similar morphology and spatial distribution, leading to the vast majority remaining unidentified. In this paper, a novel cone identification approach is proposed that is specially designed for adequately recognizing cones from different regions in high-resolution planetary images. First, due to the lack of a publicly available cone database for reference, we annotated 3681 cones according to the literature on manual interpretation and the cone information provided by the Lunar and Planetary Laboratory (IRL) in HiRISE images. Then, the cone identification problem was converted into an instance segmentation task, i.e., a cone identification approach was designed based on deep neural networks. The Feature Pyramid Network-equipped Mask R-CNN was utilized as the detection and segmentation model. Extensive experiments were conducted for fine recognition of Martian cones with HiRISE. The results show that the proposed approach achieves high performance; it especially efficiently detects multiple types of cones while generating accurate segmentation to describe the geometry contour of cones. Finally, a Martian cone dataset with deep learning-based instance segmentation (DL-MCD) was built, containing 3861 cones for exploring geological processes on the surface of Mars. [ABSTRACT FROM AUTHOR]

The integration of deep neural networks into sparse synthetic aperture radar (SAR) imaging is explored to enhance SAR imaging performance and reduce the system's sampling rate. However, the scarcity of training samples and mismatches between the training data and the SAR system pose significant challenges to the method's further development. In this paper, we propose a novel SAR imaging approach based on deep image prior powered by RED (DeepRED), enabling unsupervised SAR imaging without the need for additional training data. Initially, DeepRED is introduced as the regularization technique within the sparse SAR imaging model. Subsequently, variable splitting and the alternating direction method of multipliers (ADMM) are employed to solve the imaging model, alternately updating the magnitude and phase of the SAR image. Additionally, the SAR echo simulation operator is utilized as an observation model to enhance computational efficiency. Through simulations and real data experiments, we demonstrate that our method maintains imaging quality and system downsampling rate on par with deep-neural-network-based sparse SAR imaging but without the requirement for training data. [ABSTRACT FROM AUTHOR]

With the widespread application and functional complexity of deep neural networks (DNNs), the demand for training samples is increasing. This elevated requirement also extends to DNN-based SAR object detection. Most public SAR object detection datasets are oriented to marine targets such as ships, while data sets oriented to land targets are relatively rare, though they are an effective way to improve the land object detection capability of deep models through SAR sample generation. In this paper, a synthesis generation collaborative SAR sample augmentation framework is proposed to achieve flexible and diverse high-quality sample augmentation. First, a semantic-layout-guided image synthesis strategy is proposed to generate diverse detection samples. The issues of object location rationality and object layout diversity are also addressed. Meanwhile, a pix2pixGAN network guided by layout maps is utilized to achieve diverse background augmentation. Second, a progressive training strategy of diffusion models is proposed to achieve semantically controllable SAR sample generation to further improve the diversity of scene clutter. Finally, a sample cleaning method considering distribution migration and network filtering is employed to further improve the quality of detection samples. The experimental results show that this semantic synthesis generation method can outperform existing sample augmentation methods, leading to a comprehensive improvement in the accuracy metrics of classical detection networks. [ABSTRACT FROM AUTHOR]

The special geographical location of the polar regions increases the difficulty of modeling surface emissivity, thus the physical retrieval algorithms of the temperature and humidity profiles for microwave radiometers mainly focus on the regions between 60°S and 60°N. In this paper, the deep neural networks (DNN) and long short-term memory (LSTM) models are first implemented to retrieve atmospheric temperature and humidity profiles in real time from FY-3D/MWHTS in Arctic regions and are compared with the physical retrieval algorithm. The hyperparameters of the machine learning models are determined using the grid search and 10-fold cross-validation. Results show that, compared with the physical retrieval algorithm, the retrieval accuracies of the atmospheric temperature and humidity profiles of the DNN and LSTM models in June 2021 are higher over sea ice, and the maximum retrieval accuracies are improved by about 3.5 K and 42%. Over land, the retrieval accuracies of the atmospheric temperature profiles for the DNN and LSTM models in June 2021 are improved by about 5 K. The retrieved humidity results for these two models are not compared with the physical retrieval algorithm, which fails for the humidity profile retrieval over land. In addition, the retrieval results of the DNN-based and LSTM-based models using the independent validation data in February, April, and September are also evaluated over different surface types. The RMSEs of the retrieved temperature profiles for the two models are within 4 K, except for the near-surface, and the humidity profiles are within 25%, except for in February. The temperature profiles in September and the humidity profiles in February are somewhat reduced compared to other months because of the highly variable emissivity properties in autumn and winter. Overall results show that the machine learning method can well-evaluate the retrieval capability of FY-3D/MWHTS of the atmospheric temperature and humidity profiles in Arctic regions. [ABSTRACT FROM AUTHOR]

Accurate estimation of the maize leaf area index (LAI) and biomass is of great importance in guiding field management and early yield estimation. Physical models and traditional machine learning methods are commonly used for LAI and biomass estimation. However, these models and methods mostly rely on handcrafted features and theoretical formulas under idealized assumptions, which limits their accuracy. Deep neural networks have demonstrated great superiority in automatic feature extraction and complicated nonlinear approximation, but their application to LAI and biomass estimation has been hindered by the shortage of in situ data. Therefore, bridging the gap of data shortage and making it possible to leverage deep neural networks to estimate maize LAI and biomass is of great significance. Optical data cannot provide information in the lower canopy due to the limited penetrability, but synthetic aperture radar (SAR) data can do this, so the integration of optical and SAR data is necessary. In this paper, 158 samples from the jointing, trumpet, flowering, and filling stages of maize were collected for investigation. First, we propose an improved version of the mixup training method, which is termed mixup+, to augment the sample amount. We then constructed a novel gated Siamese deep neural network (GSDNN) based on a gating mechanism and a Siamese architecture to integrate optical and SAR data for the estimation of the LAI and biomass. We compared the accuracy of the GSDNN with those of other machine learning methods, i.e., multiple linear regression (MLR), support vector regression (SVR), random forest regression (RFR), and a multilayer perceptron (MLP). The experimental results show that without the use of mixup + , the GSDNN achieved a similar accuracy to that of the simple neural network MLP in terms of R 2 and RMSE, and this was slightly lower than those of MLR, SVR, and RFR. However, with the help of mixup + , the GSDNN achieved state-of-the-art performance ( R 2 = 0.71, 0.78, and 0.86 and RMSE = 0.58, 871.83, and 150.76 g/m 2 , for LAI, Biomass_wet, and Biomass_dry, respectively), exceeding the accuracies of MLR, SVR, RFR, and MLP. In addition, through the integration of optical and SAR data, the GSDNN achieved better accuracy in LAI and biomass estimation than when optical or SAR data alone were used. We found that the most appropriate amount of synthetic data from mixup + was five times the amount of original data. Overall, this study demonstrates that the GSDNN + mixup + has great potential for the integration of optical and SAR data with the aim of improving the estimation accuracy of the maize LAI and biomass with limited in situ data. [ABSTRACT FROM AUTHOR]

Conventional practices of bridge visual inspection present several limitations, including a tedious process of analyzing images manually to identify potential damages. Vision-based techniques, particularly Deep Convolutional Neural Networks, have been widely investigated to automatically identify, localize, and quantify defects in bridge images. However, massive datasets with different annotation levels are required to train these deep models. This paper presents a dataset of more than 6900 images featuring three common defects of concrete bridges (i.e., cracks, efflorescence, and spalling). To overcome the challenge of limited training samples, three Transfer Learning approaches in fine-tuning the state-of-the-art Visual Geometry Group network were studied and compared to classify the three defects. The best-proposed approach achieved a high testing accuracy (97.13%), combined with high F1-scores of 97.38%, 95.01%, and 97.35% for cracks, efflorescence, and spalling, respectively. Furthermore, the effectiveness of interpretable networks was explored in the context of weakly supervised semantic segmentation using image-level annotations. Two gradient-based backpropagation interpretation techniques were used to generate pixel-level heatmaps and localize defects in test images. Qualitative results showcase the potential use of interpretation maps to provide relevant information on defect localization in a weak supervision framework. [ABSTRACT FROM AUTHOR]

Portable devices play an essential role where edge computing is necessary and mobility is required (e.g., robots in agriculture within remote-sensing applications). With the increasing applications of deep neural networks (DNNs) and accelerators for edge devices, several methods and applications have been proposed for simultaneous crop and weed detection. Although preliminary studies have investigated the performance of inference time for semantic segmentation of crops and weeds in edge devices, performance degradation has not been evaluated in detail when the required optimization is applied to the model for operation in such edge devices. This paper investigates the relationship between model tuning hyperparameters to improve inference time and its effect on segmentation performance. The study was conducted using semantic segmentation model DeeplabV3 with a MobileNet backbone. Different datasets (Cityscapes, PASCAL and ADE20K) were analyzed for a transfer learning strategy. The results show that, when using a model hyperparameter depth multiplier (DM) of 0.5 and the TensorRT framework, segmentation performance mean intersection over union (mIOU) decreased by 14.7% compared to that of a DM of 1.0 and no TensorRT. However, inference time accelerated dramatically by a factor of 14.8. At an image resolution of 1296 × 966 , segmentation performance of 64% mIOU and inference of 5.9 frames per second (FPS) was achieved in Jetson Nano's device. With an input image resolution of 513 × 513 , and hyperparameters output stride OS = 32 and DM = 0.5, an inference time of 0.04 s was achieved resulting in 25 FPS. The results presented in this paper provide a deeper insight into how the performance of the semantic segmentation model of crops and weeds degrades when optimization is applied to adapt the model to run on edge devices. Lastly, an application is described for the semantic segmentation of weeds embedded in the edge device (Jetson Nano) and integrated with the robotic orchard. The results show good spraying accuracy and feasibility of the method. [ABSTRACT FROM AUTHOR]

Atmospheric effects are among the primary error sources affecting the accuracy of interferometric synthetic aperture radar (InSAR). The topography-dependent atmospheric effect is particularly noteworthy in reservoir areas for landslide monitoring utilizing InSAR, which must be effectively corrected to complete the InSAR high-accuracy measurement. This paper proposed a topography-dependent atmospheric correction method based on the Multi-Layer Perceptron (MLP) neural network model combined with topography and spatial data information. We used this proposed approach for the atmospheric correction of the interferometric pairs of Sentinel-1 images in the Baihetan dam. We contrasted the outcomes with those obtained using the generic atmospheric correction online service for InSAR (GACOS) correction and the traditional linear model correction. The results indicated that the MLP neural network model correction reduced the phase standard deviation of the Sentinel-1 interferogram by an average of 64% and nearly eliminated the phase-elevation correlation. Both comparisons outperformed the GACOS correction and the linear model correction. Through two real-world examples, we demonstrated how slopes with displacements, which were previously obscured by a significant topography-dependent atmospheric delay, could be successfully and clearly identified in the interferograms following the correction by the MLP neural network. The topography-dependent atmosphere can be better corrected using the MLP neural network model suggested in this paper. Unlike the previous model, this proposed approach could be adjusted to fit each interferogram, regardless of how much of the topography-dependent atmosphere was present. In order to improve the effectiveness of DInSAR and time-series InSAR solutions, it can be applied immediately to the interferogram to retrieve the effective displacement information that cannot be identified before the correction. [ABSTRACT FROM AUTHOR]

Cloud segmentation is a fundamental step in accurately acquiring cloud cover. However, due to the nonrigid structures of clouds, traditional cloud segmentation methods perform worse than expected. In this paper, a novel deep convolutional neural network (CNN) named MA-SegCloud is proposed for segmenting cloud images based on a multibranch asymmetric convolution module (MACM) and an attention mechanism. The MACM is composed of asymmetric convolution, depth-separable convolution, and a squeeze-and-excitation module (SEM). The MACM not only enables the network to capture more contextual information in a larger area but can also adaptively adjust the feature channel weights. The attention mechanisms SEM and convolutional block attention module (CBAM) in the network can strengthen useful features for cloud image segmentation. As a result, MA-SegCloud achieves a 96.9% accuracy, 97.0% precision, 97.0% recall, 97.0% F-score, 3.1% error rate, and 94.0% mean intersection-over-union (MIoU) on the Singapore Whole-sky Nychthemeron Image Segmentation (SWINySEG) dataset. Extensive evaluations demonstrate that MA-SegCloud performs favorably against state-of-the-art cloud image segmentation methods. [ABSTRACT FROM AUTHOR]

Vegetation net primary productivity (VNPP) is the main factor in ecosystem carbon sink function and regulation of environmental processes. However, NPP data products have data missing in some areas, which affects the availability and overall accuracy level of data. Therefore, we adopted the Factor Analysis Backpropagation neural network model (FA-BP model) to acquire a high-accuracy and high-reliability NPP result without missing or empty areas by using a series of easily accessible datasets, such as meteorological data and remote sensing data. We selected the western Sichuan region as the study area and carried out a VNPP time series estimation from 2000 to 2016. Comparative simulations also verify the accuracy of the time series estimation results: The Pearson correlation r of VNPP prediction results ranged from 0.807 to 0.917, the mean absolute error ranged from 29.1 to 38.9, the root mean square error was between 37.3 and 51.8, and the mean relative error varies from 0.10 to 0.14. Further analysis shows that the spatial pattern of estimated VNPP during the past 17 years in western Sichuan shows a decreasing trend from southeast to northwest. Besides, the VNPP time series is generally on an upward trend in this period. The increasing and decreasing areas of VNPP values in the study area accounted for 81.42% and 18.58%, respectively. Moreover, we find that temperature dominates the change of VNPP in the whole western Sichuan region. The data processing method and experimental results presented in this paper can provide a reference for accurately acquiring VNPP and related studies on natural resources and climate change. [ABSTRACT FROM AUTHOR]

As a safety-related application, visual systems based on deep neural networks (DNNs) in modern unmanned aerial vehicles (UAVs) show adversarial vulnerability when performing real-time inference. Recently, deep ensembles with various defensive strategies against adversarial samples have drawn much attention due to the increased diversity and reduced variance for their members. Aimed at the recognition task of remote sensing images (RSIs), this paper proposes to use a reactive-proactive ensemble defense framework to solve the security problem. In reactive defense, we fuse scoring functions of several classical detection algorithms with the hidden features and average output confidences from sub-models as a second fusion. In terms of proactive defense, we attempt two strategies, including enhancing the robustness of each sub-model and limiting the transferability among sub-models. In practical applications, the real-time RSIs are first input to the reactive defense part, which can detect and reject the adversarial RSIs. The accepted ones are then passed to robust recognition with a proactive defense. We conduct extensive experiments on three benchmark RSI datasets (i.e., UCM, AID, and FGSC-23). The experimental results show that the deep ensemble method of reactive and proactive defense performs very well in gradient-based attacks. The analysis of the applicable attack scenarios for each proactive ensemble defense is also helpful for this field. We also perform a case study with the whole framework in the black-box scenario, and the highest detection rate reaches 93.25%. Most of the adversarial RSIs can be rejected in advance or correctly recognized by the enhanced deep ensemble. This article is the first one to combine reactive and proactive defenses with a deep ensemble against adversarial attacks in the context of RSI recognition for DNN-based UAVs. [ABSTRACT FROM AUTHOR]

Due to the fact that point clouds are always corrupted by significant noise and large transformations, aligning two point clouds by deep neural networks is still challenging. This paper presents a semi-supervised point cloud registration (PCR) method for accurately estimating point correspondences and handling large transformations using limited prior datasets. Firstly, a modified autoencoder is introduced as the feature extraction module to extract the distinctive and robust features for the downstream registration task. Unlike optimization-based pairwise PCR strategies, the proposed method treats two point clouds as two implementations of a Gaussian mixture model (GMM), which we call latent GMM. Based on the above assumption, two point clouds can be regarded as two probability distributions. Hence, the PCR of two point clouds can be approached by minimizing the KL divergence between these two probability distributions. Then, the correspondence between the point clouds and the latent GMM components is estimated using the augmented regression network. Finally, the parameters of GMM can be updated by the correspondence and the transformation matrix can be computed by employing the weighted singular value decomposition (SVD) method. Extensive experiments conducted on both synthetic and real-world data validate the superior performance of the proposed method compared to state-of-the-art registration methods. These experiments also highlight the method's superiority in terms of accuracy, robustness, and generalization. [ABSTRACT FROM AUTHOR]