Search

Generative adversarial networks (or GANs) are a specific deep learning architecture often used for different usages, such as data generation or image-to-image translation. In recent years, this structure has gained increased popularity and has been used in different fields. One area of expertise currently in vogue is the use of GANs to produce synthetic aperture radar (SAR) data, and especially expand training datasets for SAR automatic target recognition (ATR). In effect, the complex SAR image formation makes these kind of data rich in information, leading to the use of deep networks in deep learning-based methods. Yet, deep networks also require sufficient data for training. However, contrary to optical images, we generally do not have a substantial number of available SAR images because of their acquisition and labelling cost; GANs are then an interesting tool. Concurrently, how to improve explainability for SAR ATR deep neural networks and how to make their reasoning more transparent have been increasingly explored as model opacity deteriorates trust of users. This paper aims at reviewing how GANs are used with SAR images, but also giving perspectives on how GANs could be used to improve interpretability and explainability of SAR classifiers. [ABSTRACT FROM AUTHOR]

The objectives of this paper are to investigate the trade-offs between a physically constrained neural network and a deep, convolutional neural network and to design a combined ML approach ("VarioCNN"). Our solution is provided in the framework of a cyberinfrastructure that includes a newly designed ML software, GEOCLASS-image (v1.0), modern high-resolution satellite image data sets (Maxar WorldView data), and instructions/descriptions that may facilitate solving similar spatial classification problems. Combining the advantages of the physically-driven connectionist-geostatistical classification method with those of an efficient CNN, VarioCNN provides a means for rapid and efficient extraction of complex geophysical information from submeter resolution satellite imagery. A retraining loop overcomes the difficulties of creating a labeled training data set. Computational analyses and developments are centered on a specific, but generalizable, geophysical problem: The classification of crevasse types that form during the surge of a glacier system. A surge is a glacial catastrophe, an acceleration of a glacier to typically 100–200 times its normal velocity. GEOCLASS-image is applied to study the current (2016-2024) surge in the Negribreen Glacier System, Svalbard. The geophysical result is a description of the structural evolution and expansion of the surge, based on crevasse types that capture ice deformation in six simplified classes. [ABSTRACT FROM AUTHOR]

Domain generalization (DG) aims to learn knowledge from multiple related domains to achieve a robust generalization performance in unseen target domains, which is an effective approach to mitigate domain shift in remote sensing image classification. Although the sharpness-aware minimization (SAM) method enhances DG capability and improves remote sensing image classification performance by promoting the convergence of the loss minimum to a flatter loss surface, the perturbation loss (maximum loss within the neighborhood of a local minimum) of SAM fails to accurately measure the true sharpness of the loss landscape. Furthermore, its variants often overlook gradient conflicts, thereby limiting further improvement in DG performance. In this paper, we introduce implicit sharpness-aware minimization (ISAM), a novel method that addresses the deficiencies of SAM and mitigates gradient conflicts. Specifically, we demonstrate that the discrepancy in training loss during gradient ascent or descent serves as an equivalent measure of the dominant eigenvalue of the Hessian matrix. This discrepancy provides a reliable measure for sharpness. ISAM effectively reduces sharpness and mitigates potential conflicts between gradients by implicitly minimizing the discrepancy between training losses while ensuring a sufficiently low minimum through minimizing perturbation loss. Extensive experiments and analyses demonstrate that ISAM significantly enhances the model's generalization ability on remote sensing and DG datasets, outperforming existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In recent years, with the continuous development and popularization of unmanned aerial vehicle (UAVs) technology, the surge in the number of UAVs has led to an increasingly serious problem of illegal flights. Traditional acoustic-based UAV localization techniques have limited ability to extract short-time and long-time signal features, and have poor localization performance in low signal-to-noise ratio environments. For this reason, in this paper, we propose a deep learning-based UAV localization technique in low signal-to-noise ratio environments. Specifically, on the one hand, we propose a multiple signal classification (MUSIC) pseudo-spectral normalized mean processing technique to improve the direction of arrival (DOA) performance of a traditional broadband MUSIC algorithm. On the other hand, we design a DOA estimation algorithm for UAV sound sources based on a time delay estimation neural network, which solves the problem of limited DOA resolution and the poor performance of traditional time delay estimation algorithms under low signal-to-noise ratio conditions. We verify the feasibility of the proposed method through simulation experiments and experiments in real scenarios. The experimental results show that our proposed method can locate the approximate flight path of a UAV within 20 m in a real scenario with a signal-to-noise ratio of −8 dB. [ABSTRACT FROM AUTHOR]

Hyperspectral imaging often suffers from various types of noise, including sensor non-uniformity and atmospheric disturbances. Removing multiple types of complex noise in hyperspectral images (HSIs) while preserving high fidelity in spectral dimensions is a challenging task in hyperspectral data processing. Existing methods typically focus on specific types of noise, resulting in limited applicability and an inadequate ability to handle complex noise scenarios. This paper proposes a denoising method based on a network that considers both the spatial structure and spectral differences of noise in an image data cube. The proposed network takes into account the DN value of the current band, as well as the horizontal, vertical, and spectral gradients as inputs. A multi-resolution convolutional module is employed to accurately extract spatial and spectral noise features, which are then aggregated through residual connections at different levels. Finally, the residual mixed noise is approximated. Both simulated and real case studies confirm the effectiveness of the proposed denoising method. In the simulation experiment, the average PSNR value of the denoised results reached 31.47 at a signal-to-noise ratio of 8 dB, and the experimental results on the real data set Indian Pines show that the classification accuracy of the denoised hyperspectral image (HSI) is improved by 16.31% compared to the original noisy version. [ABSTRACT FROM AUTHOR]

A huge amount of remote sensing data is acquired each day, which is transferred to image processing centers and/or to customers. Due to different limitations, compression has to be applied on-board and/or on-the-ground. This Special Issue collects 15 papers dealing with remote sensing data compression, introducing solutions for both lossless and lossy compression, analyzing the impact of compression on different processes, investigating the suitability of neural networks for compression, and researching on low complexity hardware and software approaches to deliver competitive coding performance. [ABSTRACT FROM AUTHOR]

For most of the arable land in Russia (132–137 million ha), the dominant and accurate soil information is stored in the form of map archives on paper without coordinate reference. The last traditional soil map(s) (TSM, TSMs) were created over 30 years ago. Traditional and/or archival soil map(s) (ASM, ASMs) are outdated in terms of storage formats, dates, and methods of production. The technology of constructing a multitemporal soil line (MSL) makes it possible to update ASMs and TSMs based on the processing of big remote-sensing data (RSD). To construct an MSL, the spectral characteristics of the bare soil surface (BSS) are used. The BSS on RSD is distinguished within the framework of the conceptual apparatus of the spectral neighborhood of the soil line. The filtering of big RSD is based on deep machine learning. In the course of the work, a vector georeferenced version of the ASM and an updated soil map were created based on the coefficient "C" of the MSL. The maps were verified based on field surveys (76 soil pits). The updated map is called the map of soil interpretation of the coefficient "C" (SIC "C"). The SIC "C" map has a more detailed legend compared to the ASM (7 sections/chapters instead of 5), greater accuracy (smaller errors of the first and second kind), and potential suitability for calculating soil organic matter/carbon (SOM/SOC) reserves (soil types/areals in the SIC "C" map are statistically significant are divided according to the thickness of the organomineral horizon and the content of SOM in the plowed layer). When updating, a systematic underestimation of the numbers of contours and areas of soils with manifestations of negative/degradation soil processes (slitization and erosion) on the TSM was established. In the process of updating, all three shortcomings of the ASMs/TSMs (archaic storage, dates, and methods of creation) were eliminated. The SIC "C" map is digital (thematic raster), modern, and created based on big data processing methods. For the first time, the actualization of the soil map was carried out based on the MSL characteristics (coefficient "C"). [ABSTRACT FROM AUTHOR]

Soil moisture maps are essential for hydrological, agricultural and risk assessment applications. To best meet these requirements, it is essential to develop soil moisture products at high spatial resolution, which is now made possible using the free Sentinel-1 (S1) SAR (Synthetic Aperture Radar) data. Some soil moisture retrieval techniques using S1 data relied on the use of a priori weather information in order to increase the precision of soil moisture estimates, which required access to a weather-forecasting framework. This paper presents an improved and fully autonomous solution for high-resolution soil moisture mapping in bare agricultural areas. The proposed solution derives a priori weather information directly from the original Sentinel images, thus bypassing the need for a weather forecasting framework. For soil moisture estimation, the neural network technique was implemented to ensure the optimum integration of radar information. The neural networks were trained using synthetic data generated by the modified Integral Equation Model (IEM) model and validated on real data from two study sites in France and Tunisia. The main findings showed that the use of a radar signal averaged over grids of a few km 2 in addition to radar signal at plot scale instead of a priori weather information provides good soil moisture estimations. The accuracy is even slightly better compared to the accuracy obtained using a priori weather information. [ABSTRACT FROM AUTHOR]

The rapid development of neural networks has come at the cost of increased computational complexity. Neural networks are both computationally intensive and memory intensive; as such, the minimal energy and computing power of satellites pose a challenge for automatic target recognition (ATR). Knowledge distillation (KD) can distill knowledge from a cumbersome teacher network to a lightweight student network, transferring the essential information learned by the teacher network. Thus, the concept of KD can be used to improve the accuracy of student networks. Even when learning from a teacher network, there is still redundancy in the student network. Traditional networks fix the structure before training, such that training does not improve the situation. This paper proposes a distillation sparsity training (DST) algorithm based on KD and network pruning to address the above limitations. We first improve the accuracy of the student network through KD, and then through network pruning, allowing the student network to learn which connections are essential. DST allows the teacher network to teach the pruned student network directly. The proposed algorithm was tested on the CIFAR-100, MSTAR, and FUSAR-Ship data sets, with a 50% sparsity setting. First, a new loss function for the teacher-pruned student was proposed, and the pruned student network showed a performance close to that of the teacher network. Second, a new sparsity model (uniformity half-pruning UHP) was designed to solve the problem that unstructured pruning does not facilitate the implementation of general-purpose hardware acceleration and storage. Compared with traditional unstructured pruning, UHP can double the speed of neural networks. [ABSTRACT FROM AUTHOR]

The SWOT (Surface Water Ocean Topography) mission will provide high-resolution and two-dimensional measurements of sea surface height (SSH). However, despite its unprecedented precision, SWOT's Ka-band Radar Interferometer (KaRIn) still exhibits a substantial amount of random noise. In turn, the random noise limits the ability of SWOT to capture the smallest scales of the ocean's topography and its derivatives. In that context, this paper explores the feasibility, strengths and limits of a noise-reduction algorithm based on a convolutional neural network. The model is based on a U-Net architecture and is trained and tested with simulated data from the North Atlantic. Our results are compared to classical smoothing methods: a median filter, a Lanczos kernel smoother and the SWOT de-noising algorithm developed by Gomez-Navarro et al. Our U-Net model yields better results for all the evaluation metrics: 2 mm root mean square error, sub-millimetric bias, variance reduction by factor of 44 (16 dB) and an accurate power spectral density down to 10–20 km wavelengths. We also tested various scenarios to infer the robustness and the stability of the U-Net. The U-Net always exhibits good performance and can be further improved with retraining if necessary. This robustness in simulation is very encouraging: our findings show that the U-Net architecture is likely one of the best candidates to reduce the noise of flight data from KaRIn. [ABSTRACT FROM AUTHOR]

Ground-penetrating radar (GPR) is an efficient and nondestructive geophysical method with great potential for detecting soil water content at the farmland scale. However, a key challenge in soil detection is obtaining soil water content rapidly and in real-time. In recent years, deep learning methods have become more widespread in the earth sciences, making it possible to use them for soil water content inversion from GPR data. In this paper, we propose a neural network framework GPRSW based on deep learning of GPR data. GPRSW is an end-to-end network that directly inverts volumetric soil water content (VSWC) through single-channel GPR data. Synthetic experiments show that GPRSW accurately identifies different VSWC boundaries in the model in time depth. The predicted VSWC and model fit well within 40 ns, with a maximum error after 40 ns of less than 0.10 cm3 × cm−3. To validate our method, we conducted GPR measurements at the experimental field of the Academy of Agricultural Sciences in Gongzhuling City, Jilin Province and applied GPRSW to VSWC measurements. The results show that predicted values of GPRSW match with field soil samples and are consistent with the overall trend of the TDR soil probe samples, with a maximum difference not exceeding 0.03 cm3 × cm−3. Therefore, our study shows that GPRSW has the potential to be applied to obtain soil water content from GPR data on farmland. [ABSTRACT FROM AUTHOR]

This paper investigates a novel Efficient Dual-branch Bottleneck Network (EDBNet) to perform real-time semantic segmentation tasks on mobile robot systems based on CCD camera. To remedy the non-linear connection between the input and the output, a small-scale and shallow module called the Efficient Dual-branch Bottleneck (EDB) module is established. The EDB unit consists of two branches with different dilation rates, and each branch widens the non-linear layers. This module helps to simultaneously extract local and situational information while maintaining a minimal set of parameters. Moreover, the EDBNet, which is built on the EDB unit, is intended to enhance accuracy, inference speed, and parameter flexibility. It employs dilated convolution with a high dilation rate to increase the receptive field and three downsampling procedures to maintain feature maps with superior spatial resolution. Additionally, the EDBNet uses effective convolutions and compresses the network layer to reduce computational complexity, which is an efficient technique to capture a great deal of information while keeping a rapid computing speed. Finally, using the CamVid and Cityscapes datasets, we obtain Mean Intersection over Union (MIoU) results of 68.58 percent and 71.21 percent, respectively, with just 1.03 million parameters and faster performance on a single GTX 1070Ti card. These results also demonstrate the effectiveness of the practical mobile robot system. [ABSTRACT FROM AUTHOR]

While thunderstorms can pose severe risks to property and life, forecasting remains challenging, even at short lead times, as these often arise in meta-stable atmospheric conditions. In this paper, we examine the question of how well we could perform short-term (up to 180 min) forecasts using exclusively multi-spectral satellite images and past lighting events as data. We employ representation learning based on deep convolutional neural networks in an "end-to-end" fashion. Here, a crucial problem is handling the imbalance of the positive and negative classes appropriately in order to be able to obtain predictive results (which is not addressed by many previous machine-learning-based approaches). The resulting network outperforms previous methods based on physically based features and optical flow methods (similar to operational prediction models) and generalizes across different years. A closer examination of the classifier performance over time and under masking of input data indicates that the learned model actually draws most information from structures in the visible spectrum, with infrared imaging sustaining some classification performance during the night. [ABSTRACT FROM AUTHOR]

Visual SLAM (VSLAM) has been developing rapidly due to its advantages of low-cost sensors, the easy fusion of other sensors, and richer environmental information. Traditional visionbased SLAM research has made many achievements, but it may fail to achieve wished results in challenging environments. Deep learning has promoted the development of computer vision, and the combination of deep learning and SLAM has attracted more and more attention. Semantic information, as high-level environmental information, can enable robots to better understand the surrounding environment. This paper introduces the development of VSLAM technology from two aspects: traditional VSLAM and semantic VSLAM combined with deep learning. For traditional VSLAM, we summarize the advantages and disadvantages of indirect and direct methods in detail and give some classical VSLAM open-source algorithms. In addition, we focus on the development of semantic VSLAM based on deep learning. Starting with typical neural networks CNN and RNN, we summarize the improvement of neural networks for the VSLAM system in detail. Later, we focus on the help of target detection and semantic segmentation for VSLAM semantic information introduction. We believe that the development of the future intelligent era cannot be without the help of semantic technology. Introducing deep learning into the VSLAM system to provide semantic information can help robots better perceive the surrounding environment and provide people with higher-level help. [ABSTRACT FROM AUTHOR]

The detection of degraded soil distribution areas is an urgent task. It is difficult and very time consuming to solve this problem using ground methods. The modeling of degradation processes based on digital elevation models makes it possible to construct maps of potential degradation, which may differ from the actual spatial distribution of degradation. The use of remote sensing data (RSD) for soil degradation detection is very widespread. Most often, vegetation indices (indicative botany) have been used for this purpose. In this paper, we propose a method for constructing soil maps based on a multi-temporal analysis of the bare soil surface (BSS). It is an alternative method to the use of vegetation indices. The detection of the bare soil surface was carried out using the spectral neighborhood of the soil line (SNSL) technology. For the automatic recognition of BSS on each RSD image, computer vision based on deep machine learning (neural networks) was used. A dataset of 244 BSS distribution masks on 244 Landsat 4, 5, 7, and 8 scenes over 37 years was developed. Half of the dataset was used as a training sample (Landsat path/row 173/028). The other half was used as a test sample (Landsat path/row 174/027). Binary masks were sufficient for recognition. For each RSD pixel, value "1" was set when determining the BSS. In the absence of BSS, value "0" was set. The accuracy of the machine prediction of the presence of BSS was 75%. The detection of degradation was based on the average long-term spectral characteristics of the RED and NIR bands. The coefficient Cmean, which is the distance of the point with the average long-term values of RED and NIR from the origin of the spectral plane RED/NIR, was calculated as an integral characteristic of the mean long-term values. Higher long-term average values of spectral brightness served as indicators of the spread of soil degradation. To test the method of constructing soil degradation maps based on deep machine learning, an acceptance sample of 133 Landsat scenes of path/row 173/026 was used. On the territory of the acceptance sample, ground verifications of the maps of the coefficient Cmean were carried out. Ground verification showed that the values of this coefficient make it possible to estimate the content of organic matter in the plow horizon (R2 = 0.841) and the thickness of the humus horizon (R2 = 0.8599). In total, 80 soil pits were analyzed on an area of 649 ha on eight agricultural fields. Type I error (false positive) of degradation detection was 17.5%, and type II error (false negative) was 2.5%. During the determination of the presence of degradation by ground methods, 90% of the ground data coincided with the detection of degradation from RSD. Thus, the quality of machine learning for BSS recognition is sufficient for the construction of soil degradation maps. The SNSL technology allows us to create maps of soil degradation based on the long-term average spectral characteristics of the BSS. [ABSTRACT FROM AUTHOR]

Soil moisture mapping at a high spatial resolution is very important for several applications in hydrology, agriculture and risk assessment. With the arrival of the free Sentinel data at high spatial and temporal resolutions, the development of soil moisture products that can better meet the needs of users is now possible. In this context, the main objective of the present paper is to develop an operational approach for soil moisture mapping in agricultural areas at a high spatial resolution over bare soils, as well as soils with vegetation cover. The developed approach is based on the synergic use of radar and optical data. A neural network technique was used to develop an operational method for soil moisture estimates. Three inversion SAR (Synthetic Aperture Radar) configurations were tested: (1) VV polarization; (2) VH polarization; and (3) both VV and VH polarization, all in addition to the NDVI information extracted from optical images. Neural networks were developed and validated using synthetic and real databases. The results showed that the use of a priori information on the soil moisture condition increases the precision of the soil moisture estimates. The results showed that VV alone provides better accuracy on the soil moisture estimates than VH alone. In addition, the use of both VV and VH provides similar results, compared to VV alone. In conclusion, the soil moisture could be estimated in agricultural areas with an accuracy of approximately 5 vol % (volumetric unit expressed in percent). Better results were obtained for soil with a moderate surface roughness (for root mean surface height between 1 and 3 cm). The developed approach could be applied for agricultural plots with an NDVI lower than 0.75. [ABSTRACT FROM AUTHOR]

We present a study on the effectiveness of using varying data sizes to transfer crop type classification models from one year to the next, emphasizing the balance between data sufficiency and model accuracy. The significance of crop detection through satellite imaging lies in its potential to enhance agricultural productivity and resource management. Machine learning, particularly techniques like long short-term memory (LSTM) models, has become instrumental in interpreting these satellite data due to its predictive accuracy and adaptability. However, the direct application of models trained in one year to subsequent years poses challenges due to variations in environmental conditions and agricultural practices. Fine-tuning pre-existing models is a prevalent strategy to overcome these temporal discrepancies, though it necessitates a careful evaluation of the quantity and relevance of new data. This study explores the cost–benefit of fine-tuning existing models versus developing new ones based on the quantity of new data, utilizing LSTM models for their transferability and practicality in agricultural applications. Experiments conducted using satellite data from farms in southern Alberta reveal that smaller datasets, with fewer than 25 fields per class, can effectively fine-tune models for accurate interannual classification, while larger datasets are more conducive to training new models. This poses a key challenge in optimizing data usage for crop classification, straddling the line between data sufficiency and computational efficiency. The findings offer valuable insights for optimizing data use in crop classification, benefiting both academic research and practical agricultural applications. [ABSTRACT FROM AUTHOR]

Precision viticulture systems are essential for enhancing traditional intensive viticulture, achieving high-quality results, and minimizing costs. This study explores the integration of Unmanned Aerial Vehicles (UAVs) and artificial intelligence in precision viticulture, focusing on vine detection and vineyard zoning. Vine detection employs the YOLO (You Only Look Once) deep learning algorithm, achieving a remarkable 90% accuracy by analysing UAV imagery with various spectral ranges from various phenological stages. Vineyard zoning, achieved through the application of the K-means algorithm, incorporates geospatial data such as the Normalized Difference Vegetation Index (NDVI) and the assessment of nitrogen, phosphorus, and potassium content in leaf blades and petioles. This approach enables efficient resource management tailored to each zone's specific needs. The research aims to develop a decision-support model for precision viticulture. The proposed model demonstrates a high vine detection accuracy and defines management zones with variable weighting factors assigned to each variable while preserving location information, revealing significant differences in variables. The model's advantages lie in its rapid results and minimal data requirements, offering profound insights into the benefits of UAV application for precise vineyard management. This approach has the potential to expedite decision making, allowing for adaptive strategies based on the unique conditions of each zone. [ABSTRACT FROM AUTHOR]

Terrace detection and ridge extraction from high-resolution remote sensing imagery are crucial for soil conservation and grain production on sloping land. Traditional methods use low-to-medium resolution images, missing detailed features and lacking automation. Terrace detection and ridge extraction are closely linked, with each influencing the other's outcomes. However, most studies address these tasks separately, overlooking their interdependence. This research introduces a cutting-edge, multi-scale, and multi-task deep learning framework, termed DTRE-Net, designed for comprehensive terrace information extraction. This framework bridges the gap between terrace detection and ridge extraction, executing them concurrently. The network incorporates residual networks, multi-scale fusion modules, and multi-scale residual correction modules to enhance the model's robustness in feature extraction. Comprehensive evaluations against other deep learning-based semantic segmentation methods using GF-2 terraced imagery from two distinct areas were undertaken. The results revealed intersection over union (IoU) values of 85.18% and 86.09% for different terrace morphologies and 59.79% and 73.65% for ridges. Simultaneously, we have confirmed that the connectivity of results is improved when employing multi-task learning for ridge extraction compared to directly extracting ridges. These outcomes underscore DTRE-Net's superior capability in the automation of terrace and ridge extraction relative to alternative techniques. [ABSTRACT FROM AUTHOR]

This study investigates the potential of Sentinel-1 (S1) multi-temporal data for the early-season mapping of the rapeseed crop. Additionally, we explore the effectiveness of limiting the portion of a considered time series to map rapeseed fields. To this end, we conducted a quantitative analysis to assess several temporal windows (periods) spanning different phases of the rapeseed phenological cycle in the following two scenarios relating to the availability or constraints of providing ground samples for different years: (i) involving the same year for both training and the test, assuming the availability of ground samples for each year; and (ii) evaluating the temporal transferability of the classifier, considering the constraints of ground sampling. We employed two different classification methods that are renowned for their high performance in land cover mapping: the widely adopted random forest (RF) approach and a deep learning-based convolutional neural network, specifically the InceptionTime algorithm. To assess the classification outcomes, four evaluation metrics (recall, precision, F1 score, and Kappa) were employed. Using S1 time series data covering the entire rapeseed growth cycle, the tested algorithms achieved F1 scores close to 95% on same-year training and testing, and 92.0% when different years were used, both algorithms demonstrated robust performance. For early rapeseed detection within a two-month window post-sowing, RF and InceptionTime achieved F1 scores of 67.5% and 77.2%, respectively, and 79.8% and 88.9% when extended to six months. However, in the context of temporal transferability, both classifiers exhibited mean F1 scores below 50%. Notably, a 5-month time series, covering key growth stages such as stem elongation, inflorescence emergence, and fruit development, yielded a mean F1 score close to 95% for both algorithms when trained and tested in the same year. In the temporal transferability scenario, RF and InceptionTime achieved mean F1 scores of 92.0% and 90.0%, respectively, using a 5-month time series. Our findings underscore the importance of a concise S1 time series for effective rapeseed mapping, offering advantages in data storage and processing time. Overall, the study establishes the robustness of RF and InceptionTime in rapeseed detection scenarios, providing valuable insights for agricultural applications. [ABSTRACT FROM AUTHOR]

Domain generalization (DG) aims to learn knowledge from multiple related domains to achieve a robust generalization performance in unseen target domains, which is an effective approach to mitigate domain shift in remote sensing image classification. Although the sharpness-aware minimization (SAM) method enhances DG capability and improves remote sensing image classification performance by promoting the convergence of the loss minimum to a flatter loss surface, the perturbation loss (maximum loss within the neighborhood of a local minimum) of SAM fails to accurately measure the true sharpness of the loss landscape. Furthermore, its variants often overlook gradient conflicts, thereby limiting further improvement in DG performance. In this paper, we introduce implicit sharpness-aware minimization (ISAM), a novel method that addresses the deficiencies of SAM and mitigates gradient conflicts. Specifically, we demonstrate that the discrepancy in training loss during gradient ascent or descent serves as an equivalent measure of the dominant eigenvalue of the Hessian matrix. This discrepancy provides a reliable measure for sharpness. ISAM effectively reduces sharpness and mitigates potential conflicts between gradients by implicitly minimizing the discrepancy between training losses while ensuring a sufficiently low minimum through minimizing perturbation loss. Extensive experiments and analyses demonstrate that ISAM significantly enhances the model’s generalization ability on remote sensing and DG datasets, outperforming existing state-of-the-art methods.

Generative adversarial networks (or GANs) are a specific deep learning architecture often used for different usages, such as data generation or image-to-image translation. In recent years, this structure has gained increased popularity and has been used in different fields. One area of expertise currently in vogue is the use of GANs to produce synthetic aperture radar (SAR) data, and especially expand training datasets for SAR automatic target recognition (ATR). In effect, the complex SAR image formation makes these kind of data rich in information, leading to the use of deep networks in deep learning-based methods. Yet, deep networks also require sufficient data for training. However, contrary to optical images, we generally do not have a substantial number of available SAR images because of their acquisition and labelling cost; GANs are then an interesting tool. Concurrently, how to improve explainability for SAR ATR deep neural networks and how to make their reasoning more transparent have been increasingly explored as model opacity deteriorates trust of users. This paper aims at reviewing how GANs are used with SAR images, but also giving perspectives on how GANs could be used to improve interpretability and explainability of SAR classifiers.

In recent years, with the continuous development and popularization of unmanned aerial vehicle (UAVs) technology, the surge in the number of UAVs has led to an increasingly serious problem of illegal flights. Traditional acoustic-based UAV localization techniques have limited ability to extract short-time and long-time signal features, and have poor localization performance in low signal-to-noise ratio environments. For this reason, in this paper, we propose a deep learning-based UAV localization technique in low signal-to-noise ratio environments. Specifically, on the one hand, we propose a multiple signal classification (MUSIC) pseudo-spectral normalized mean processing technique to improve the direction of arrival (DOA) performance of a traditional broadband MUSIC algorithm. On the other hand, we design a DOA estimation algorithm for UAV sound sources based on a time delay estimation neural network, which solves the problem of limited DOA resolution and the poor performance of traditional time delay estimation algorithms under low signal-to-noise ratio conditions. We verify the feasibility of the proposed method through simulation experiments and experiments in real scenarios. The experimental results show that our proposed method can locate the approximate flight path of a UAV within 20 m in a real scenario with a signal-to-noise ratio of −8 dB.

The objectives of this paper are to investigate the trade-offs between a physically constrained neural network and a deep, convolutional neural network and to design a combined ML approach (“VarioCNN”). Our solution is provided in the framework of a cyberinfrastructure that includes a newly designed ML software, GEOCLASS-image (v1.0), modern high-resolution satellite image data sets (Maxar WorldView data), and instructions/descriptions that may facilitate solving similar spatial classification problems. Combining the advantages of the physically-driven connectionist-geostatistical classification method with those of an efficient CNN, VarioCNN provides a means for rapid and efficient extraction of complex geophysical information from submeter resolution satellite imagery. A retraining loop overcomes the difficulties of creating a labeled training data set. Computational analyses and developments are centered on a specific, but generalizable, geophysical problem: The classification of crevasse types that form during the surge of a glacier system. A surge is a glacial catastrophe, an acceleration of a glacier to typically 100–200 times its normal velocity. GEOCLASS-image is applied to study the current (2016-2024) surge in the Negribreen Glacier System, Svalbard. The geophysical result is a description of the structural evolution and expansion of the surge, based on crevasse types that capture ice deformation in six simplified classes.

Building-change detection underpins many important applications, especially in the military and crisis-management domains. Recent methods used for change detection have shifted towards deep learning, which depends on the quality of its training data. The assembly of large-scale annotated satellite imagery datasets is therefore essential for global building-change surveillance. Existing datasets almost exclusively offer near-nadir viewing angles. This limits the range of changes that can be detected. By offering larger observation ranges, the scroll imaging mode of optical satellites presents an opportunity to overcome this restriction. This paper therefore introduces S2Looking, a building-change-detection dataset that contains large-scale side-looking satellite images captured at various off-nadir angles. The dataset consists of 5000 bitemporal image pairs of rural areas and more than 65,920 annotated instances of changes throughout the world. The dataset can be used to train deep-learning-based change-detection algorithms. It expands upon existing datasets by providing (1) larger viewing angles; (2) large illumination variances; and (3) the added complexity of rural images. To facilitate the use of the dataset, a benchmark task has been established, and preliminary tests suggest that deep-learning algorithms find the dataset significantly more challenging than the closest-competing near-nadir dataset, LEVIR-CD+. S2Looking may therefore promote important advances in existing building-change-detection algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we present neural network methods for predicting uncertainty in atmospheric remote sensing. These include methods for solving the direct and the inverse problem in a Bayesian framework. In the first case, a method based on a neural network for simulating the radiative transfer model and a Bayesian approach for solving the inverse problem is proposed. In the second case, (i) a neural network, in which the output is the convolution of the output for a noise-free input with the input noise distribution; and (ii) a Bayesian deep learning framework that predicts input aleatoric and model uncertainties, are designed. In addition, a neural network that uses assumed density filtering and interval arithmetic to compute uncertainty is employed for testing purposes. The accuracy and the precision of the methods are analyzed by considering the retrieval of cloud parameters from radiances measured by the Earth Polychromatic Imaging Camera (EPIC) onboard the Deep Space Climate Observatory (DSCOVR). [ABSTRACT FROM AUTHOR]

Satellite image time series (SITS) is a sequence of satellite images that record a given area at several consecutive times. The aim of such sequences is to use not only spatial information but also the temporal dimension of the data, which is used for multiple real-world applications, such as classification, segmentation, anomaly detection, and prediction. Several traditional machine learning algorithms have been developed and successfully applied to time series for predictions. However, these methods have limitations in some situations, thus deep learning (DL) techniques have been introduced to achieve the best performance. Reviews of machine learning and DL methods for time series prediction problems have been conducted in previous studies. However, to the best of our knowledge, none of these surveys have addressed the specific case of works using DL techniques and satellite images as datasets for predictions. Therefore, this paper concentrates on the DL applications for SITS prediction, giving an overview of the main elements used to design and evaluate the predictive models, namely the architectures, data, optimization functions, and evaluation metrics. The reviewed DL-based models are divided into three categories, namely recurrent neural network-based models, hybrid models, and feed-forward-based models (convolutional neural networks and multi-layer perceptron). The main characteristics of satellite images and the major existing applications in the field of SITS prediction are also presented in this article. These applications include weather forecasting, precipitation nowcasting, spatio-temporal analysis, and missing data reconstruction. Finally, current limitations and proposed workable solutions related to the use of DL for SITS prediction are also highlighted. [ABSTRACT FROM AUTHOR]

Water depth estimation in seaports is essential for effective port management. This paper presents an empirical approach for water depth determination from satellite imagery through the integration of multiple datasets and machine learning algorithms. The implementation details of the proposed approach are provided and compared against different existing machine learning algorithms with a single training set. For a single training set and a single machine learning method, our analysis shows that the proposed depth estimation method provides a better root-mean-square error (RMSE) and a higher coefficient of determination (R2) under turbid water conditions, with overall RMSE and R2 improvements of 1 cm and 0.7, respectively. The developed method may be employed in monitoring dredging activities, especially in areas with polluted water, mud and/or a high sediment content. [ABSTRACT FROM AUTHOR]

The images of the Sentinel-2 constellation can help the verification process of farmers' declarations, providing, among other things, accurate spatial explicit maps of the agricultural land cover. The aim of the study is to design, develop, and evaluate two deep learning (DL) architectures tailored for agricultural land cover and crop type mapping. The focus is on a detailed class scheme encompassing fifteen distinct classes, utilizing Sentinel-2 imagery acquired on a monthly basis throughout the year. The study's geographical scope covers a diverse rural area in North Greece, situated within southeast Europe. These architectures are a Temporal Convolutional Neural Network (CNN) and a combination of a Recurrent and a 2D Convolutional Neural Network (R-CNN), and their accuracy is compared to the well-established Random Forest (RF) machine learning algorithm. The comparative approach is not restricted to simply presenting the results given by classification metrics, but it also assesses the uncertainty of the classification results using an entropy measure and the spatial distribution of the classification errors. Furthermore, the issue of sampling strategy for the extraction of the training set is highlighted, targeting the efficient handling of both the imbalance of the dataset and the spectral variability of instances among classes. The two developed deep learning architectures performed equally well, presenting an overall accuracy of 90.13% (Temporal CNN) and 90.18% (R-CNN), higher than the 86.31% overall accuracy of the RF approach. Finally, the Temporal CNN method presented a lower entropy value (6.63%), compared both to R-CNN (7.76%) and RF (28.94%) methods, indicating that both DL approaches should be considered for developing operational EO processing workflows. [ABSTRACT FROM AUTHOR]

Accurate location information has significant commercial and economic value as they are widely used in intelligent manufacturing, material localization and smart homes. Magnetic sequence-based approaches show great promise mainly due to their pervasiveness and stability. However, existing geomagnetic indoor localization methods are facing the problems of location ambiguity and feature extraction deficiency, which will lead to large localization errors. To address these issues, we propose a coarse-to-fine geomagnetic indoor localization method based on deep learning. First, a multidimensional geomagnetic feature extraction method is presented which can extract magnetic features from spatial and temporal aspects. Then, a hierarchical deep neural network model is devised to extract more accurate geomagnetic information and corresponding location clues for more accurate localization. Finally, localization is achieved through a particle filter combined with IMU localization. To evaluate the performance of the proposed methods, we carried out several experiments at three trial paths with two heterogeneous devices, Vivo X30 and Huawei Mate30. Experimental results demonstrate that the proposed algorithm can achieve more accurate localization performance than the state-of-the-art methods. Meanwhile, the proposed algorithm has low cost and good pervasiveness for different devices. [ABSTRACT FROM AUTHOR]

The performance of the underwater acoustic (UWA) orthogonal frequency division multiplexing (OFDM) system is often restrained by time-varying channels with large delays. The existing frequency domain equalizers do not work well because of the high complexity and difficulty of finding the real-time signal-to-noise ratio. To solve these problems, we propose a low-complexity neural network (NN)-based scheme for joint equalization and detection. A simple NN structure is built to yield the detected symbols with the joint input of the segmented channel response and received symbol. The coherence bandwidth is investigated to find the optimal hyperparameters. By being completely trained offline with real channels, the proposed detector is applied independently in both simulations and sea trials. The results show that the proposed detector outperforms the ZF and MMSE equalizers and extreme learning machine (ELM)-based detectors in both the strongly reflected channels of the pool and time-variant channels of the shallow sea. The complexity of the proposed network is lower than the MMSE and ELM-based receiver. [ABSTRACT FROM AUTHOR]

Over one hundred spatiotemporal fusion algorithms have been proposed, but convolutional neural networks trained with large amounts of data for spatiotemporal fusion have not shown significant advantages. In addition, no attention has been paid to whether fused images can be used for change detection. These two issues are addressed in this work. A new dataset consisting of nine pairs of images is designed to benchmark the accuracy of neural networks using one-pair spatiotemporal fusion with neural-network-based models. Notably, the size of each image is significantly larger compared to other datasets used to train neural networks. A comprehensive comparison of the radiometric, spectral, and structural losses is made using fourteen fusion algorithms and five datasets to illustrate the differences in the performance of spatiotemporal fusion algorithms with regard to various sensors and image sizes. A change detection experiment is conducted to test if it is feasible to detect changes in specific land covers using the fusion results. The experiment shows that convolutional neural networks can be used for one-pair spatiotemporal fusion if the sizes of individual images are adequately large. It also confirms that the spatiotemporally fused images can be used for change detection in certain scenes. [ABSTRACT FROM AUTHOR]

Data-driven deep learning has been well applied in radar target detection. However, the performance of the detection network is severely degraded when the detection scene changes, since the trained network with the data from one scene is not suitable for another scene with different data distribution. In order to address this problem, an adaptive network detector combined with scene classification is proposed in this paper. Aiming at maximizing the posterior probability of the feature vectors, the scene classification network is arranged to control the output ratio of a group of detection sub-networks. Due to the uncertainty of classification error rate in traditional machine learning, the classifier with a controllable false alarm rate is constructed. In addition, a new network training strategy, which freezes the parameters of the scene classification network and selectively fine-tunes the parameters of detection sub-networks, is proposed for the adaptive network structure. Comprehensive experiments are carried out to demonstrate that the proposed method guarantees a high detection probability when the detection scene changes. Compared with some classical detectors, the adaptive network detector shows better performance. [ABSTRACT FROM AUTHOR]

Recently, convolutional neural networks have been successfully applied to lossy image compression. End-to-end optimized autoencoders, possibly variational, are able to dramatically outperform traditional transform coding schemes in terms of rate-distortion trade-off; however, this is at the cost of a higher computational complexity. An intensive training step on huge databases allows autoencoders to learn jointly the image representation and its probability distribution, possibly using a non-parametric density model or a hyperprior auxiliary autoencoder to eliminate the need for prior knowledge. However, in the context of on board satellite compression, time and memory complexities are submitted to strong constraints. The aim of this paper is to design a complexity-reduced variational autoencoder in order to meet these constraints while maintaining the performance. Apart from a network dimension reduction that systematically targets each parameter of the analysis and synthesis transforms, we propose a simplified entropy model that preserves the adaptability to the input image. Indeed, a statistical analysis performed on satellite images shows that the Laplacian distribution fits most features of their representation. A complex non parametric distribution fitting or a cumbersome hyperprior auxiliary autoencoder can thus be replaced by a simple parametric estimation. The proposed complexity-reduced autoencoder outperforms the Consultative Committee for Space Data Systems standard (CCSDS 122.0-B) while maintaining a competitive performance, in terms of rate-distortion trade-off, in comparison with the state-of-the-art learned image compression schemes. [ABSTRACT FROM AUTHOR]

At present, convolutional neural networks (CNN) have been widely used in building extraction from remote sensing imagery (RSI), but there are still some bottlenecks. On the one hand, there are so many parameters in the previous network with complex structure, which will occupy lots of memories and consume much time during training process. On the other hand, low-level features extracted by shallow layers and abstract features extracted by deep layers of artificial neural network cannot be fully fused, which leads to an inaccurate building extraction from RSI. To alleviate these disadvantages, a dense residual neural network (DR-Net) was proposed in this paper. DR-Net uses a deeplabv3+Net encoder/decoder backbone, in combination with densely connected convolution neural network (DCNN) and residual network (ResNet) structure. Compared with deeplabv3+net (containing about 41 million parameters) and BRRNet (containing about 17 million parameters), DR-Net contains about 9 million parameters; So, the number of parameters reduced a lot. The experimental results for both the WHU Building Dataset and Massachusetts Building Dataset, DR-Net show better performance in building extraction than other two state-of-the-art methods. Experiments on WHU building data set showed that Intersection over Union (IoU) increased by 2.4% and F1 score increased by 1.4%; in terms of Massachusetts Building Dataset, IoU increased by 3.8% and F1 score increased by 2.9%. [ABSTRACT FROM AUTHOR]

Soil degradation processes are widespread on agricultural land. Ground-based methods for detecting degradation require a lot of labor and time. Remote methods based on the analysis of vegetation indices can significantly reduce the volume of ground surveys. Currently, machine learning methods are increasingly being used to analyze remote sensing data. In this paper, the task is set to apply deep machine learning methods and methods of vegetation indices calculation to automate the detection of areas of soil degradation development on arable land. In the course of the work, a method was developed for determining the location of degraded areas of soil cover on arable fields. The method is based on the use of multi-temporal remote sensing data. The selection of suitable remote sensing data scenes is based on deep machine learning. Deep machine learning was based on an analysis of 1028 scenes of Landsats 4, 5, 7 and 8 on 530 agricultural fields. Landsat data from 1984 to 2019 was analyzed. Dataset was created manually for each pair of "Landsat scene"/"agricultural field number"(for each agricultural field, the suitability of each Landsat scene was assessed). Areas of soil degradation were calculated based on the frequency of occurrence of low NDVI values over 35 years. Low NDVI values were calculated separately for each suitable fragment of the satellite image within the boundaries of each agricultural field. NDVI values of one-third of the field area and lower than the other two-thirds were considered low. During testing, the method gave 12.5% of type I errors (false positive) and 3.8% of type II errors (false negative). Independent verification of the method was carried out on six agricultural fields on an area of 713.3 hectares. Humus content and thickness of the humus horizon were determined in 42 ground-based points. In arable land degradation areas identified by the proposed method, the probability of detecting soil degradation by field methods was 87.5%. The probability of detecting soil degradation by ground-based methods outside the predicted regions was 3.8%. The results indicate that deep machine learning is feasible for remote sensing data selection based on a binary dataset. This eliminates the need for intermediate filtering systems in the selection of satellite imagery (determination of clouds, shadows from clouds, open soil surface, etc.). Direct selection of Landsat scenes suitable for calculations has been made. It allows automating the process of constructing soil degradation maps. [ABSTRACT FROM AUTHOR]

Change detection for high-resolution remote sensing images is more and more widespread in the application of monitoring the Earth's surface. However, on the one hand, the ground truth could facilitate the distinction between changed and unchanged areas, but it is hard to acquire them. On the other hand, due to the complexity of remote sensing images, it is difficult to extract features of difference, let alone the construction of the classification model that performs change detection based on the features of difference in each pixel pair. Aiming at these challenges, this paper proposes a weakly supervised change detection method based on edge mapping and Stacked Denoising Auto-Encoders (SDAE) network called EM-SDAE. We analyze the difference in edge maps of bi-temporal remote sensing images to acquire part of the ground truth at a relatively low cost. Moreover, we design a neural network based on SDAE with a deep structure, which extracts the features of difference so as to efficiently classify changed and unchanged regions after being trained with the ground truth. In our experiments, three real sets of high-resolution remote sensing images are employed to validate the high efficiency of our proposed method. The results show that accuracy can even reach up to 91.18% with our method. In particular, compared with the state-of-the-art work (e.g., IR-MAD, PCA-k-means, CaffeNet, USFA, and DSFA), it improves the Kappa coefficient by 27.19% on average. [ABSTRACT FROM AUTHOR]

In this study, a new technique is proposed to retrieve temperature and relative humidity profiles under clear sky conditions in the Arctic region based on the artificial neural network (ANN) algorithm using Fengyun-3D (FY-3D) vertical atmospheric sounder suit (VASS: HIRAS, MWTS-II, and MWHS-II) observations. This technology combines infrared (IR) and microwave (MW) observations to improve retrieval accuracy in the middle and low troposphere by reducing the sensitivity of the neural networks (NNs) to cloud coverage. The approach was compared against other methods available in the literature on retrieving profiles only from FY-3D/HIRAS data. Furthermore, its retrieval performance was tested by comparing the NNs' prediction accuracy versus the corresponding FY-3D/VASS and Aqua/AIRS L2 products. The results showed that: (1) NNs retrieval accuracy is higher during the warm season and over the ocean; (2) the retrieval accuracy of NNs has been significantly improved compared with satellite L2 products; (3) referring to radiosonde observations, the retrieval accuracy of NNs below 600 hPa is effectively improved by adding the information of the MW channel, especially on land where cloud clearing is more difficult. The root mean square error (RMSE) of temperature and relative humidity in the cold season were reduced by 0.3 K and 2%, respectively. The advanced NNs proposed herein offer a more stable retrieval performance compared with NNs built only by FY-3D/HIRAS data. The study results indicated the potential value in time and space domain of the NN algorithm in retrieving temperature and relative humidity profiles of the Arctic region from FY-3D/VASS observations under clear-sky conditions. All in all, this work enhances our knowledge towards improving operational use of FY-3D satellite data in the Arctic region. [ABSTRACT FROM AUTHOR]

In real estate appraisal, especially of residential buildings, one of the primary evaluation parameters is the property's usable area. When determining the property price, Polish appraisers use data from comparable transactions included in the Real Estate Price Register (REPR), which is highly incomplete, especially regarding properties' usable areas. This incompleteness renders the identification of comparable transactions challenging and may lead to incorrect prediction of the property price. We address this challenge by applying machine learning methods to estimate the usable area of buildings with gable roofs based only on their topographic data, which is widely available in Poland in the Database of Topographic Objects (BDOT10k) of Light Detection and Ranging (LiDAR) origin. We show that three features are enough to make accurate predictions of the usable area: the covered area, the building's height, and the number of stories optionally. A neural network trained on buildings from architectural bureaus reached a 4% median percentage error on the same source and 15% on the real buildings from the city of Koszalin, Poland. Therefore, the proposed method can be applied by appraisers to estimate the usable area of buildings with known transaction prices and solve the problem of finding comparable properties for appraisal. [ABSTRACT FROM AUTHOR]

Radar images suffer from the impact of sidelobes. Several sidelobe-suppressing methods including the convolutional neural network (CNN)-based one has been proposed. However, the point spread function (PSF) in the radar images is sometimes spatially variant and affects the performance of the CNN. We propose the spatial-variant convolutional neural network (SV-CNN) aimed at this problem. It will also perform well in other conditions when there are spatially variant features. The convolutional kernels of the CNN can detect motifs with some distinctive features and are invariant to the local position of the motifs. This makes the convolutional neural networks widely used in image processing fields such as image recognition, handwriting recognition, image super-resolution, and semantic segmentation. They also perform well in radar image enhancement. However, the local position invariant character might not be good for radar image enhancement, when features of motifs (also known as the point spread function in the radar imaging field) vary with the positions. In this paper, we proposed an SV-CNN with spatial-variant convolution kernels (SV-CK). Its function is illustrated through a special application of enhancing the radar images. After being trained using radar images with position-codings as the samples, the SV-CNN can enhance the radar images. Because the SV-CNN reads information of the local position contained in the position-coding, it performs better than the conventional CNN. The advance of the proposed SV-CNN is tested using both simulated and real radar images. [ABSTRACT FROM AUTHOR]

In this paper, we investigate the feasibility of automatic small object detection, such as vehicles and vessels, in satellite imagery with a spatial resolution between 0.3 and 0.5 m. The main challenges of this task are the small objects, as well as the spread in object sizes, with objects ranging from 5 to a few hundred pixels in length. We first annotated 1500 km2, making sure to have equal amounts of land and water data. On top of this dataset we trained and evaluated four different single-shot object detection networks: YOLOV2, YOLOV3, D-YOLO and YOLT, adjusting the many hyperparameters to achieve maximal accuracy. We performed various experiments to better understand the performance and differences between the models. The best performing model, D-YOLO, reached an average precision of 60% for vehicles and 66% for vessels and can process an image of around 1 Gpx in 14 s. We conclude that these models, if properly tuned, can thus indeed be used to help speed up the workflows of satellite data analysts and to create even bigger datasets, making it possible to train even better models in the future. [ABSTRACT FROM AUTHOR]

The availability of freshwater is becoming a global concern. Because agricultural consumption has been increasing steadily, the mapping of irrigated areas is key for supporting the monitoring of land use and better management of available water resources. In this paper, we propose a method to automatically detect and map center pivot irrigation systems using U-Net, an image segmentation convolutional neural network architecture, applied to a constellation of PlanetScope images from the Cerrado biome of Brazil. Our objective is to provide a fast and accurate alternative to map center pivot irrigation systems with very high spatial and temporal resolution imagery. We implemented a modified U-Net architecture using the TensorFlow library and trained it on the Google cloud platform with a dataset built from more than 42,000 very high spatial resolution PlanetScope images acquired between August 2017 and November 2018. The U-Net implementation achieved a precision of 99% and a recall of 88% to detect and map center pivot irrigation systems in our study area. This method, proposed to detect and map center pivot irrigation systems, has the potential to be scaled to larger areas and improve the monitoring of freshwater use by agricultural activities. [ABSTRACT FROM AUTHOR]

The Data Interpolating Empirical Orthogonal Functions (DINEOF) method has demonstrated usability and accuracy for filling spatial gaps in remote sensing datasets. In this study, we conducted the reconstruction of the chlorophyll-a concentration (Chl-a) data using a convolutional neural networks model called Data-Interpolating Convolutional Auto-Encoder (DINCAE), and we compared its performance with that of DINEOF. Furthermore, the cloud-free sea surface temperature (SST) was used as a phytoplankton dynamics predictor for the Chl-a reconstruction. Finally, four reconstruction schemes were implemented: DINCAE (Chl-a only), DINCAE (Chl-a and SST), DINEOF (Chl-a only), and DINEOF (Chl-a and SST), denoted rec1, rec2, rec3, and rec4 respectively. To quantitatively evaluate the accuracy of these reconstruction schemes, both the cross-validation and in situ data were used. The study domain was chosen to be the Northern South China Sea (SCS) and West Philippine Sea (WPS), bounded by 115–125°E and 16–24°N to test the model performance for the reconstruction of Chl-a under different Chl-a controlling mechanisms. The in situ validation showed that rec1 performs best among the four reconstruction schemes, and that adding SST into the Chl-a reconstruction cannot improve the reconstruction results. However, for cross validation, adding SST can slightly improve spatial distributions of the root mean square error (RMSE) between the reconstructed data and the original data, especially over the SCS continental shelf. Furthermore, the potential of DINCAE prediction is confirmed in this paper; thus, the trained DINCAE model can be re-applied to reconstruct other missing data, and more importantly, it can also be re-trained using the reconstructed data, thereby further improving reconstruction results. Another consideration is efficiency; with similar reconstruction conditions, DINCAE is 5–10 times faster than DINEOF. [ABSTRACT FROM AUTHOR]

The long-term spectral characteristics of the bare soil surface (BSS) in the BLUE, GREEN, RED, NIR, SWIR1, and SWIR2 Landsat spectral bands are poorly studied. Most often, the RED and NIR spectral bands are used to analyze the spatial heterogeneity of the soil cover; in our opinion, it is outmoded and seems unreasonable. The study of multi-temporal spectral characteristics requires the processing of big remote sensing data based on artificial intelligence in the form of convolutional neural networks. The analysis of BSS belongs to the direct methods of analysis of the soil cover. Soil degradation can be detected by ground methods (field reconnaissance surveys), modeling, or digital methods, and based on the remote sensing data (RSD) analysis. Ground methods are laborious, and modeling gives indirect results. RSD analysis can be based on the principles of calculation of vegetation indices (VIs) and on the BSS identification. The calculation of VIs also provides indirect information about the soil cover through the state of vegetation. BSS analysis is a direct method for analyzing soil cover heterogeneity. In this work, the informativeness of the long-term (37 years) average spectral characteristics of the BLUE, GREEN, RED, NIR, SWIR1 and SWIR2 bands of the Landsat 4–8 satellites for detecting areas of soil degradation with recognition of the BSS using deep machine learning methods was estimated. The objects of study are the spectral characteristics of kastanozems (dark chestnut soils) in the south of Russia in the territory of the Morozovsky district of the Rostov region. Soil degradation in this area is mainly caused by erosion. The following methods were used: retrospective monitoring of soil and land cover, deep machine learning using convolutional neural networks, and cartographic analysis. Six new maps of the average long-term spectral brightness of the BSS have been obtained. The information content of the BSS for six spectral bands has been verified on the basis of ground surveys. The informativeness was determined by the percentage of coincidences of degradation facts identified during the RSD analysis, and those determined in the field. It has been established that the spectral bands line up in the following descending order of information content: RED, NIR, GREEN, BLUE, SWIR1, SWIR2. The accuracy of degradation maps by band was determined as: RED—84.6%, NIR—82.9%, GREEN—78.0%, BLUE—78.0%, SWIR1—75.5%, SWIR2—62.2%. [ABSTRACT FROM AUTHOR]

Different satellite images may consist of variable numbers of channels which have different resolutions, and each satellite has a unique revisit period. For example, the Landsat-8 satellite images have 30 m resolution in their multispectral channels, the Sentinel-2 satellite images have 10 m resolution in the pan-sharp channel, and the National Agriculture Imagery Program (NAIP) aerial images have 1 m resolution. In this study, we propose a simple yet effective arithmetic deep model for multimodal temporal remote sensing image fusion. The proposed model takes both low- and high-resolution remote sensing images at t 1 together with low-resolution images at a future time t 2 from the same location as inputs and fuses them to generate high-resolution images for the same location at t 2 . We propose an arithmetic operation applied to the low-resolution images at the two time points in feature space to take care of temporal changes. We evaluated the proposed model on three modality pairs for multimodal temporal image fusion, including downsampled WorldView-2/original WorldView-2, Landsat-8/Sentinel-2, and Sentinel-2/NAIP. Experimental results show that our model outperforms traditional algorithms and recent deep learning-based models by large margins in most scenarios, achieving sharp fused images while appropriately addressing temporal changes. [ABSTRACT FROM AUTHOR]

Accurately predicting total electron content (TEC) during geomagnetic storms is still a challenging task for ionospheric models. In this work, a neural-network (NN)-based model is proposed which predicts relative TEC with respect to the preceding 27-day median TEC, during storm time for the European region (with longitudes 30°W–50°E and latitudes 32.5°N–70°N). The 27-day median TEC (referred to as median TEC), latitude, longitude, universal time, storm time, solar radio flux index F10.7, global storm index SYM-H and geomagnetic activity index Hp30 are used as inputs and the output of the network is the relative TEC. The relative TEC can be converted to the actual TEC knowing the median TEC. The median TEC is calculated at each grid point over the European region considering data from the last 27 days before the storm using global ionosphere maps (GIMs) from international GNSS service (IGS) sources. A storm event is defined when the storm time disturbance index Dst drops below 50 nanotesla. The model was trained with storm-time relative TEC data from the time period of 1998 until 2019 (2015 is excluded) and contains 365 storms. Unseen storm data from 33 storm events during 2015 and 2020 were used to test the model. The UQRG GIMs were used because of their high temporal resolution (15 min) compared to other products from different analysis centers. The NN-based model predictions show the seasonal behavior of the storms including positive and negative storm phases during winter and summer, respectively, and show a mixture of both phases during equinoxes. The model's performance was also compared with the Neustrelitz TEC model (NTCM) and the NN-based quiet-time TEC model, both developed at the German Aerospace Agency (DLR). The storm model has a root mean squared error (RMSE) of 3.38 TEC units (TECU), which is an improvement by 1.87 TECU compared to the NTCM, where an RMSE of 5.25 TECU was found. This improvement corresponds to a performance increase by 35.6%. The storm-time model outperforms the quiet-time model by 1.34 TECU, which corresponds to a performance increase by 28.4% from 4.72 to 3.38 TECU. The quiet-time model was trained with Carrington averaged TEC and, therefore, is ideal to be used as an input instead of the GIM derived 27-day median. We found an improvement by 0.8 TECU which corresponds to a performance increase by 17% from 4.72 to 3.92 TECU for the storm-time model using the quiet-time-model predicted TEC as an input compared to solely using the quiet-time model. [ABSTRACT FROM AUTHOR]

This study investigates the feasibility of applying monoplotting to video data from a security camera and image data from an uncrewed aircraft system (UAS) survey to create a mapping product which overlays traffic flow in a university parking lot onto an aerial orthomosaic. The framework, titled VirtuaLot, employs a previously defined computer-vision pipeline which leverages Darknet for vehicle detection and tests the performance of various object tracking algorithms. Algorithmic object tracking is sensitive to occlusion, and monoplotting is applied in a novel way to efficiently extract occluding features from the video using a digital surface model (DSM) derived from the UAS survey. The security camera is also a low fidelity model not intended for photogrammetry with unstable interior parameters. As monoplotting relies on static camera parameters, this creates a challenging environment for testing its effectiveness. Preliminary results indicate that it is possible to manually monoplot between aerial and perspective views with high degrees of transition tilt, achieving coordinate transformations between viewpoints within one deviation of vehicle short and long axis measurements throughout 70.5% and 99.6% of the study area, respectively. Attempted automation of monoplotting on video was met with limited success, though this study offers insight as to why and directions for future work on the subject. [ABSTRACT FROM AUTHOR]

Hyperspectral imaging often suffers from various types of noise, including sensor non-uniformity and atmospheric disturbances. Removing multiple types of complex noise in hyperspectral images (HSIs) while preserving high fidelity in spectral dimensions is a challenging task in hyperspectral data processing. Existing methods typically focus on specific types of noise, resulting in limited applicability and an inadequate ability to handle complex noise scenarios. This paper proposes a denoising method based on a network that considers both the spatial structure and spectral differences of noise in an image data cube. The proposed network takes into account the DN value of the current band, as well as the horizontal, vertical, and spectral gradients as inputs. A multi-resolution convolutional module is employed to accurately extract spatial and spectral noise features, which are then aggregated through residual connections at different levels. Finally, the residual mixed noise is approximated. Both simulated and real case studies confirm the effectiveness of the proposed denoising method. In the simulation experiment, the average PSNR value of the denoised results reached 31.47 at a signal-to-noise ratio of 8 dB, and the experimental results on the real data set Indian Pines show that the classification accuracy of the denoised hyperspectral image (HSI) is improved by 16.31% compared to the original noisy version.

For most of the arable land in Russia (132–137 million ha), the dominant and accurate soil information is stored in the form of map archives on paper without coordinate reference. The last traditional soil map(s) (TSM, TSMs) were created over 30 years ago. Traditional and/or archival soil map(s) (ASM, ASMs) are outdated in terms of storage formats, dates, and methods of production. The technology of constructing a multitemporal soil line (MSL) makes it possible to update ASMs and TSMs based on the processing of big remote-sensing data (RSD). To construct an MSL, the spectral characteristics of the bare soil surface (BSS) are used. The BSS on RSD is distinguished within the framework of the conceptual apparatus of the spectral neighborhood of the soil line. The filtering of big RSD is based on deep machine learning. In the course of the work, a vector georeferenced version of the ASM and an updated soil map were created based on the coefficient “C” of the MSL. The maps were verified based on field surveys (76 soil pits). The updated map is called the map of soil interpretation of the coefficient “C” (SIC “C”). The SIC “C” map has a more detailed legend compared to the ASM (7 sections/chapters instead of 5), greater accuracy (smaller errors of the first and second kind), and potential suitability for calculating soil organic matter/carbon (SOM/SOC) reserves (soil types/areals in the SIC “C” map are statistically significant are divided according to the thickness of the organomineral horizon and the content of SOM in the plowed layer). When updating, a systematic underestimation of the numbers of contours and areas of soils with manifestations of negative/degradation soil processes (slitization and erosion) on the TSM was established. In the process of updating, all three shortcomings of the ASMs/TSMs (archaic storage, dates, and methods of creation) were eliminated. The SIC “C” map is digital (thematic raster), modern, and created based on big data processing methods. For the first time, the actualization of the soil map was carried out based on the MSL characteristics (coefficient “C”).

Multisource remote sensing data acquisition has been increased in the last years due to technological improvements and decreased acquisition cost of remotely sensed data and products. This study attempts to fuse different types of prospection data acquired from dissimilar remote sensors and explores new ways of interpreting remote sensing data obtained from archaeological sites. Combination and fusion of complementary sensory data does not only increase the detection accuracy but it also increases the overall performance in respect to recall and precision. Moving beyond the discussion and concerns related to fusion and integration of multisource prospection data, this study argues their potential (re)use based on Bayesian Neural Network (BNN) fusion models. The archaeological site of Vésztő-Mágor Tell in the eastern part of Hungary was selected as a case study, since ground penetrating radar (GPR) and ground spectral signatures have been collected in the past. GPR 20 cm depth slices results were correlated with spectroradiometric datasets based on neural network models. The results showed that the BNN models provide a global correlation coefficient of up to 73%—between the GPR and the spectroradiometric data—for all depth slices. This could eventually lead to the potential re-use of archived geo-prospection datasets with optical earth observation datasets. A discussion regarding the potential limitations and challenges of this approach is also included in the paper. [ABSTRACT FROM AUTHOR]