Search

Supervised deep learning algorithms have recently achieved state-of-the-art performance in the classification, segmentation and analysis of 3D LiDAR point cloud data in a wide-range of applications and environments. One of the main downsides of deep learning-based approaches is the need for extensive training datasets, i.e. LiDAR point clouds that have been annotated for target tasks by human experts. One strategy for addressing this issue is the use of simulated/synthetic data (with automatically generated annotations) for training models which can then be deployed on real target data/environments. This paper explores using synthetic data of realistic forest trees to train deep learning models for tree part segmentation from real forest LiDAR data. We develop a new pipeline for generating high-fidelity simulated LiDAR scans of synthetic forest trees and combine this with an unsupervised domain adaptation strategy to adapt models trained on synthetic data to LiDAR data captured in real forest environments.Models were trained for semantic segmentation of tree parts using a PointNet++ architecture and evaluated across a range of medium to high-resolution laser scanning datasets collected across both ground-based and aerial platforms in multiple forest environments. Results of our work indicated that models trained on our synthetic data pipeline were competitive with models trained on real data, in particular when real data came from non-target sites, and our unsupervised domain adaptation method further improved performance. Our approach has implications for reducing the burden required in manual human expert annotation of large LiDAR datasets required to achieve high-performance from deep learning methods for forest analysis. The use of synthetically-trained models shown here provides a potential way to reduce the barriers to the use of deep learning in large-scale forest analysis, with implications to applications ranging from forest inventories to scaling-up in-situ forest phenotyping.

Leveraging ground-annotated data for scene analysis on unmanned aerial vehicles (UAVs) can lead to valuable real-world applications. However, existing unsupervised domain adaptive (UDA) methods primarily focus on domain confusion, which raises conflicts among training data if there is a huge domain shift caused by variations in observation perspectives or locations. To illustrate this problem, we present a ground-to-UAV fire segmentation method as a novel benchmark to verify typical UDA methods, and propose an effective framework, Colour-Mix, to boost the performance of the segmentation method equivalent to the fully supervised level. First, we identify domain-invariant fire features by deriving fire-discriminating components (u*VS) defined in colour spaces Lu*v*, YUV, and HSV. Notably, we devise criteria to combine components that are beneficial for integrating colour signals into deep-learning training, thus significantly improving the generalisation abilities of the framework without resorting to UDA techniques. Second, we perform class-specific mixing to eliminate irrelevant background content on the ground scenario and enrich annotated fire samples for the UAV imagery. Third, we propose to disentangle the feature encoding for different domains and use class-specific mixing as robust training signals for the target domain. The framework is validated on the drone-captured dataset, Flame, by using the combined ground-level source datasets, Street Fire and Corsica Wildfires. The code is available at https://github.com/Rui-Zhou-2/Colour-Mix.

Several industrial and commercial bulk material management applications rely on accurate, current stockpile volume estimation. Proximal imaging and LiDAR sensing modalities can be used to derive stockpile volume estimates in outdoor and indoor storage facilities. Among available imaging and LiDAR sensing modalities, the latter is more advantageous for indoor storage facilities due to its ability to capture scans under poor lighting conditions. Evaluating volumes from such sensing modalities requires the pose (i.e., position and orientation) parameters of the used sensors relative to a common reference frame. For outdoor facilities, a Global Navigation Satellite System (GNSS) combined with an Inertial Navigation System (INS) can be used to derive the sensors’ pose relative to a global reference frame. For indoor facilities, GNSS signal outages will not allow for such capability. Prior research has developed strategies for establishing the sensor position and orientation for stockpile volume estimation while relying on multi-beam spinning LiDAR units. These approaches are feasible due to the large range and Field of View (FOV) of such systems that can capture the internal surfaces of indoor storage facilities.The mechanical movement of multi-beam spinning LiDAR units together with the harsh conditions within indoor facilities (e.g., excessive humidity, wide range of temperature variation, dust, and corrosive environment in deicing salt storage facilities) limit the use of such systems. With the increasing availability of solid-state LiDAR units, there is an interest in exploring their potential for stockpile volume estimation. Despite their higher robustness to harsh conditions, solid-state LiDAR units have shorter distance measurement range and limited FOV when compared with multi-beam spinning LiDAR. This research presents a strategy for the extrinsic calibration (i.e., estimating the relative pose parameters) of installed solid-state LiDAR units inside stockpile storage facilities. The extrinsic calibration is made possible using deployed spherical targets and a complete, reference scan of the facility from another LiDAR sensing modality. The proposed research introduces strategies for: 1) automated extraction of the spherical targets; 2) automated matching of these targets in the solid-state LiDAR and reference scans using invariant relationships among them; and 3) coarse-to-fine estimation of the calibration parameters. Experimental results in several facilities have shown the feasibility of using the proposed methodology to conduct the extrinsic calibration and volume evaluation with an error percentage less than 3.5% even with occlusion percentages reaching up to 50%.

Terrestrial Radar Interferometry (TRI) is widely adopted in geomonitoring applications due to its capability to precisely observe surface displacements along the line of sight, among other key characteristics. As its deployment grows, TRI is also increasingly used to monitor smaller and more dispersed geological phenomena, where the challenge is their precise localization in 3d space if the pose of the radar interferometer is not known beforehand. To tackle this challenge, we introduce a semi-automatic target-based georeferencing method for precisely aligning TRI data with 3d point clouds obtained using long-range Terrestrial Laser Scanning (TLS). To facilitate this, we developed a multi-modal corner reflector (mmCR) that serves as a common reference point recognizable by both technologies, and we accompanied it with a semi-automatic data-processing pipeline, including the algorithms for precise center estimation. Experimental validation demonstrated that the corner reflector can be localized within the TLS data with a precision of 3–5 cm and within the TRI data with 1–2 dm. The targets were deployed in a realistic geomonitoring scenario to evaluate the implemented workflow and the achievable quality of georeferencing. The post-georeferencing mapping uncertainty was found to be on a decimeter level, matching the state-of-the-art results using dedicated targets and achieving more than an order of magnitude lower uncertainty than the existing data-driven approaches. In contrast to the existing target-based approaches, our results were achieved without laborious visual data inspection and manual target detection and on significantly larger distances, surpassing 2 km. The use of the developed mmCR and its associated data-processing pipeline extends beyond precise georeferencing of TRI imagery to TLS point clouds, allowing for alternatively georeferencing using total stations, mapping quality evaluation as well as on-site testing and calibrating TRI systems within the application environment.

The progressing industrialization of oceans mandates reliable, accurate and automatable subsea survey methods. Close-range photogrammetry is a promising discipline, which is frequently applied by archaeologists, fish-farmers, and the offshore energy industry. This paper presents a robust approach for the reliable detection and identification of photogrammetric markers in subsea images. The proposed method is robust to severe image degradation, which is frequently observed in underwater images due to turbidity, light absorption, and optical aberrations. This is the first step towards a highly automated work-flow for single-camera underwater photogrammetry. The newly developed approach comprises several machine learning models, which are trained by 10,122 real-world subsea images, showing a total of 338,301 photogrammetric markers. The performance is evaluated using an object detection metrics, and through a comparison with the commercially available software Metashape by Agisoft. Metashape delivers satisfactory results when the image quality is good. In images with strong noise, haze or little light, only the novel approach retrieves sufficient information for a high degree of automation of the subsequent bundle adjustment. While the need for offshore personnel and the time-to-results decreases, the robustness of the survey increases.

Forest diebacks pose a major threat to global ecosystems. Identifying and mapping both living and dead trees is crucial for understanding the causes and implementing effective management strategies. This study explores the efficacy of Mask R–CNN for automated forest dieback monitoring. The method detects individual trees, delineates their crowns, and classifies them as alive or dead. We evaluated the approach using aerial imagery and canopy height models in the Harz Mountains, Germany, a region severely affected by forest dieback. To assess the model's ability to track changes over time, we applied it to images from three separate flight campaigns (2009, 2016, and 2022). This evaluation considered variations in acquisition dates, cameras, post-processing techniques, and image tilting. Forest changes were analyzed based on the detected trees' number, spatial distribution, and height. A comprehensive accuracy assessment demonstrated the Mask R–CNN's robust performance, with precision scores ranging from 0.80 to 0.88 and F1-scores from 0.88 to 0.91. These results confirm the model's ability to generalize across diverse image acquisition conditions. While minor changes were observed between 2009 and 2016, the period between 2016 and 2022 witnessed substantial dieback, with a 64.57% loss of living trees. Notably, taller trees appeared to be particularly affected. This study highlights Mask R–CNN's potential as a valuable tool for automated forest dieback monitoring. It enables efficient detection, delineation, and classification of both living and dead trees, providing crucial data for informed forest management practices.

3D reconstruction is a long-standing research topic in the photogrammetric and computer vision communities; although a plethora of open-source and commercial solutions for 3D reconstruction have been released in the last few years, several open challenges and limitations still exist. Undoubtedly, deep learning algorithms have demonstrated great potential in several remote sensing tasks, including image-based 3D reconstruction. State-of-the-art monocular and stereo algorithms leverage deep learning techniques and achieve increased performance in depth estimation and 3D reconstruction. However, one of the limitations of such methods is that they highly rely on large training sets that are often tedious to obtain; even when available, they typically refer to indoor, close-range scenarios and low-resolution images. Especially while considering UAV (Unmanned Aerial Vehicle) scenarios, such data are not available and domain adaptation is not a trivial challenge. To fill this gap, the UAV-based multi-sensor dataset for geospatial research (UseGeo - https://usegeo.fbk.eu/home) is introduced in this paper. It contains both image and LiDAR data and aims to support relevant research in photogrammetry and computer vision with a useful training set for both stereo and monocular 3D reconstruction algorithms. In this regard, the dataset provides ground truth data for both point clouds and depth maps. In addition, UseGeo can be also a valuable dataset for other tasks such as feature extraction and matching, aerial triangulation, or image and LiDAR co-registration. The paper introduces the UseGeo dataset and validates some state-of-the-art algorithms to assess their usability for both monocular and multi-view 3D reconstruction.

Real-time object detection and tracking is an active area of aerial remote sensing research that enables many environmental and ecological monitoring and preservation applications. Despite the development of several solutions tailored for these specific applications, trade-offs between cost efficiency and feature richness persist. This paper proposes a lightweight, low-cost, and modular approach to real-time object detection and instance tracking, enabling a wide gamut of use cases. By integrating real-time object detection models with affordable embedded hardware, we present a system that uses image metadata to perform geolocation on detected objects, enabling real-time applications due to minimal computational overhead. This algorithm generates cleaner ’areas of interest’ based on geolocated detections filtered by a clustering algorithm to remove false positives. In our findings, this proved a viable solution with real-time processing speeds and GPS positioning accuracy within a meter. While there is room for improvement, our proposed pipeline represents a significant step forward in lowering the costs involved with applying computer vision to conservation applications.

Depth estimation and 3D model reconstruction from aerial imagery is an important task in photogrammetry, remote sensing, and computer vision. To compare the performance of different image-based approaches, this study presents a benchmark for UAV-based aerial imagery using the UseGeo dataset. The contributions include the release of various evaluation routines on GitHub, as well as a comprehensive comparison of baseline approaches, such as methods for offline multi-view 3D reconstruction resulting in point clouds and triangle meshes, online multi-view depth estimation, as well as single-image depth estimation using self-supervised deep learning. With the release of our evaluation routines, we aim to provide a universal protocol for the evaluation of depth estimation and 3D reconstruction methods on the UseGeo dataset. The conducted experiments and analyses show that each method excels in a different category: the depth estimation from COLMAP outperforms that of the other approaches, ACMMP achieves the lowest error and highest completeness for point clouds, while OpenMVS produces triangle meshes with the lowest error. Among the online methods for depth estimation, the approach from the Plane-Sweep Library outperforms the FaSS-MVS approach, while the latter achieves the lowest processing time. And even though the particularly challenging nature of the dataset and the small amount of training data leads to a significantly higher error in the results of the self-supervised single-image depth estimation approach, it outperforms all other approaches in terms of processing time and frame rate. In our evaluation, we have also considered modern learning-based approaches that can be used for image-based 3D reconstruction, such as NeRFs. However, due to the significantly lower quality of the resulting 3D models, we have only included a qualitative comparison between NeRF-based and conventional approaches in the scope of this work.

Predicting crop yield using deep learning (DL) and remote sensing is a promising technique in agriculture. In smallholder agriculture (

Various methods have been developed to assign pollen to its botanical origin. They range from technically complex approaches to the less precise but sophisticated chromatic assessment, in which the pollen colors are used for identification. However, a common challenge lies in the similarity of colors of pollen from different plant species. The advent of camera-based bee monitoring systems has sparked renewed interest in classifying pollen based on color and offers potential advances for honey bee biomonitoring. Despite the promise of improved sensor accuracy, a critical examination of whether color diversity within a single species may be the primary limiting factor has been lacking. Our comprehensive analysis, which includes over 85,000 corbicular pollen from 30 major pollen species, shows that the average color variation within each species is distinguishable to a human observer, similar to the difference between two dissimilar colors. From today's perspective, the considerable color variation within a single pollen source makes the use of color alone to classify pollen impractical. When picking a single pollen color from the entire dataset, we report a correct pollen type classification rate of 67 %. The accuracy was highly dependent on the type and ranged from 0 % for rare types with common colors to 99 % for distinct colors. The large color dispersion within species highlights the need for complementary methods to improve the accuracy and reliability of color-based pollen identification in biomonitoring applications.

In recent years, there has been a growing emphasis on assessing and ensuring the quality of horticultural and agricultural produce. Traditional methods involving field measurements, investigations, and statistical analyses are labour-intensive, time-consuming, and costly. As a solution, Hyperspectral Imaging (HSI) has emerged as a non-destructive and environmentally friendly technology. HSI has gained significant popularity as a new technology, particularly for its promising applications in remote sensing, notably in agriculture. However, classifying HSI data is highly complex because it involves several challenges, such as the excessive redundancy of spectral bands, scarcity of training samples, and the intricate non-linear relationship between spatial positions and spectral bands. Notably, Deep Learning (DL) techniques have demonstrated remarkable efficacy in various HSI analysis tasks, including those within agriculture. As interest continues to surge in leveraging HSI data for agricultural applications through DL approaches, a pressing need exists for a comprehensive survey that can effectively navigate researchers through the significant strides achieved and the future promising research directions in this domain. This literature review diligently compiles, analyzes, and discusses recent endeavours employing DL methodologies. These methodologies encompass a spectrum of approaches, ranging from Autoencoders (AE) to Convolutional Neural Networks (CNN) (in 1D, 2D, and 3D configurations), Recurrent Neural Networks (RNN), Deep Belief Networks (DBN), Generative Adversarial Networks (GAN), Transfer Learning (TL), Semi-Supervised Learning (SSL), Few-Shot Learning (FSL) and Active Learning (AL). These approaches are tailored to address the unique challenges posed by agricultural HSI analysis. This review evaluates and discusses the performance exhibited by these diverse approaches. To this end, the efficiency of these approaches has been rigorously analyzed and discussed based on the results of the state-of-the-art papers on widely recognized land cover datasets. Github repository.

Real-time semantic segmentation of point clouds has increasing importance in applications related to 3D city modelling and mapping, automated inventory of forests, autonomous driving and mobile robotics. Current state-of-the-art point cloud semantic segmentation methods rely heavily on the availability of 3D laser scanning data. This is problematic in regards of low-latency, real-time applications that use data from high-precision mobile laser scanners, as those are typically 2D line scanning devices. In this study, we experiment with real-time semantic segmentation of high-density multispectral point clouds collected from 2D line scanners in urban environments using encoder - decoder convolutional neural network architectures. We introduce a rasterized multi-scan input format that can be constructed exclusively from the raw (non-georeferenced profiles) 2D laser scanner measurement stream without odometry information. In addition, we investigate the impact of multispectral data on the segmentation accuracy. The dataset used for training, validation and testing was collected with multispectral FGI AkhkaR4-DW backpack laser scanning system operating at the wavelengths of 905 nm and 1550 nm, and consists in total of 228 million points (39 583 scans). The data was divided into 13 classes that represent various targets in urban environments. The results show that the increased spatial context of the multi-scan format improves the segmentation performance on the single-wavelength lidar dataset from 45.4 mIoU (a single scan) to 62.1 mIoU (24 consecutive scans). In the multispectral point cloud experiments we achieved a 71 % and 28 % relative increase in the segmentation mIoU (43.5 mIoU) as compared to the purely single-wavelength reference experiments, in which we achieved 25.4 mIoU (905 nm) and 34.1 mIoU (1550 nm). Our findings show that it is possible to semantically segment 2D line scanner data with good results by combining consecutive scans without the need for odometry information. The results also serve as motivation for developing multispectral mobile laser scanning systems that can be used in challenging urban surveys.

CNES is currently carrying out a Phase A study to assess the feasibility of a future hyperspectral imaging sensor (10 m spatial resolution) combined with a panchromatic camera (2.5 m spatial resolution). This mission focuses on both high spatial and spectral resolution requirements, as inherited from previous French studies such as HYPEX, HYPXIM, and BIODIVERSITY. To meet user requirements, cost, and instrument compactness constraints, CNES asked the French hyperspectral Mission Advisory Group (MAG), representing a broad French scientific community, to provide recommendations on spectral sampling, particularly in the Short Wave InfraRed (SWIR) for various applications.This paper presents the tests carried out with the aim of defining the optimal spectral sampling and spectral resolution in the SWIR domain for quantitative estimation of physical variables and classification purposes. The targeted applications are geosciences (mineralogy, soil moisture content), forestry (tree species classification, leaf functional traits), coastal and inland waters (bathymetry, water column, bottom classification in shallow water, coastal habitat classification), urban areas (land cover), industrial plumes (aerosols, methane and carbon dioxide), cryosphere (specific surface area, equivalent black carbon concentration), and atmosphere (water vapor, carbon dioxide and aerosols). All the products simulated in this exercise used the same CNES end-to-end processing chain, with realistic instrument parameters, enabling easy comparison between applications. 648 simulations were carried out with different spectral strategies, radiometric calibration performances and signal-to-noise Ratios (SNR): 24 instrument configurations × 25 datasets (22 images + 3 spectral libraries).The results show that spectral sampling up to 20 nm in the SWIR range is sufficient for most applications. However, 10 nm spectral sampling is recommended for applications based on specific absorption bands such as mineralogy, industrial plumes or atmospheric gases. In addition, a slight performance loss is generally observed when radiometric calibration accuracy decreases, with a few exceptions in bathymetry and in the cryosphere for which the observed performance is severely degraded. Finally, most applications can be achieved with a realistic SNR, with the exception of bathymetry, shallow water classification, as well as carbon dioxide and methane estimation, which require the optimistic SNR level tested. On the basis of these results, CNES is currently evaluating the best compromise for designing the future hyperspectral sensor to meet the objectives of priority applications.

Tightly-coupled sensor orientation, i.e. the simultaneous processing of temporal (GNSS and raw inertial) and spatial (image and lidar) constraints in a common adjustment, has demonstrated significant improvement in the quality of attitude determination with small inertial sensors. This is particularly beneficial in kinematic laser scanning on lightweight aerial platforms, such as drones, which employ direct sensor orientation for the spatial interpretation of laser vectors. In this study, previously reported preliminary results are extended to assess the gain in accuracy of sensor orientation through leveraging all available spatio-temporal constraints in a dynamic network i) with a commercial IMU for drones and ii) with simultaneous processing of raw-observations of several low-quality IMUs. Additionally, we evaluate the influence of different types of spatial constraints (image 2D and point-cloud 3D tie-points) and flight geometries (with and without a cross flight line). We present the newly implemented estimation of confidence levels and compare those with the observed residual errors. The empirical evidence demonstrates that the use of spatial constraints increases the attitude accuracy of the derived trajectory by a factor of 2–3, both for the commercial and low-quality IMUs, while at the same time reducing the dispersion of geo-referencing errors, resulting in a considerably more precise and self-coherent geo-referenced point-cloud. We further demonstrate that the use of image constraints (additionally to lidar constraints) stabilizes the in-flight lidar boresight estimation by a factor of 3–10, establishing the feasibility of such estimation even in the absence of special calibration patterns or calibration targets.

Scalable and transferable methods for generating reliable reference data for automated remote sensing approaches are crucial, especially for mapping complex Earth surface processes such as gully erosion in low-populated and inaccessible areas. As an alternative for the labour-intense in-situ authoritative mapping, collaborative approaches enable volunteers to generate redundant independent geoinformation by digitising Earth observation imagery. We face the challenge of mapping the complex gully outlines integrating multi-user contributions of the same gully network. Comparing Sentinel 2, Bing Aerial, and unoccupied aerial vehicle orthophoto base maps, we examine the volunteered geographic information process and multi-contribution integration using Kalman filtering and machine learning to segment a gully border in a remote area in northwestern Namibia. The Kalman filtering integrates the different lines finding a smoothed solution, and a Random Forest model is used to identify mapping conditions and terrain features as key predictors for evaluating contributors' digitising quality. Assessing results with expert-based reference data, we identify ten contributions as optimal, yielding root mean square distance values of 19.1 m, 15.9 m and 16.6 m, and variability of 2.0 m, 4.2 m and 3.8 m (root mean square distance standard deviation) for Sentinel 2, Bing Aerial, and unoccupied aerial vehicle orthophoto, respectively. Eliminating the lowest performing contributions for Sentinel 2 using a Random Forest regression-based quality indicator improves the accuracy by up to 35% in the root mean square distance compared to a random selection, and up to 54% compared to a supervised remote sensing classification. Results for Sentinel 2 show that low slope, low terrain ruggedness index, and high normalised difference vegetation index values are correlated to high spatial mapping deviations, with Pearson correlation coefficients of −0.61, −0.5, and 0.18, respectively. Our approach is a powerful alternative for authoritative mapping of morphologically complex environmental phenomena and can provide independent reference data for supervised automatic remote sensing analysis.

This paper introduces methods for monitoring rock slope movements in Alpine environments based on terrestrial images. The first method is a photogrammtric point cloud-based deformation analysis, relying on M3C2. Although effective in identifying large changes, the method has a tendency to underestimate smaller-scale movements. A feature-based method is presented to address this limitation, using SIFT features to track keypoints in images from different epochs. These automatically detected 3D vectors offer high spatial density and enable small-scale movement detection in the order of a few millimeters. The results are incorporated into a deformation analysis that allows statistically based conclusions about the ongoing movements. The workflow relies on georegistration using Ground Control Points. To investigate the possibility of avoiding these points, a registration method based on the ICP algorithm and M3C2 is tested. The study utilizes data from an active landslide site at Hochvogel Mountain in the Alps, analyzing changes and deformations from 2018 to 2021, revealing an average motion of 75 mm.

This paper examines the effects of implementing relative orientation constraints on bundle adjustment, as well as provides a full derivation of the Jacobian matrix for such an adjustment, that can be used to facilitate other implementations of bundle adjustment with constrained cameras. We present empirical evidence demonstrating improved accuracy and reduced computational load when these constraints are imposed.

In the view of climate change, understanding and managing effects on coastal areas and adjacent cities is essential. Permanent Laser Scanning (PLS) is a successful technique to not only observe notably sandy coasts incidentally or once every year, but (nearly) continuously over extended periods of time. The collected point cloud observations form a 4D point cloud data set representing the evolution of the coast provide the opportunity to assess change processes at high level of detail. For an exemplary location in Noordwijk, The Netherlands, three years of hourly point clouds were acquired on a 1 km long section of a typical Dutch urban sandy beach. Often, the so-called level of detection is used to assess point cloud differences from two epochs. To explicitly incorporate the temporal dimension of the height estimates from the point cloud data set, we revisit statistical testing theory. We apply multiple hypothesis testing on elevation time series in order to identify different coastal processes, like aeolian sand transport or bulldozer works. We then estimate the minimal detectable bias for different alternative hypotheses, to quantify the minimal elevation change that can be estimated from the PLS observations over a certain period of time. Additionally, we analyse potential error sources and influences on the elevation estimations and provide orders of magnitudes and possible ways to deal with them. Finally we conclude that elevation time series from a long term PLS data set are a suitable input to identify aeolian sand transport with the help of multiple hypothesis testing. In our example case, slopes of 0.032 m/day and sudden changes of 0.031 m can be identified with statistical power of 80% and with 95% significance in 24-h time series on the upper beach. In the intertidal area the presented method allows to classify daily elevation time series over one month according to the dominating model (sudden change or linear trend) in either eroding or accreting behaviour.

The TriMetrogon Aerial (TMA) archive is an archive of historical images of Antarctica taken by the US Navy between 1940 and 2000 with analogue cameras. The analysis of such historic data can give a view of Antarctica's glaciers predating modern satellite imagery and provide unique insights into the long-term impact of changing climate conditions with essential validation data for climate modelling. However, the lack of semantic information for these images presents a challenge for large-scale computer-driven analysis.Such information can be added to the data using semantic segmentation, but traditional algorithms fail on these scanned historical grayscale images, due to varying image quality, lack of colour information and artefacts in the images. To address this, we present a deep-learning-based U-net workflow. Our approach includes creating training data by pre-processing and labelling the raw images. Furthermore, different versions of the U-net are trained to optimize its hyperparameters and augmentation methods. With the optimal hyper-parameters and augmentation methods, a final model has been trained for a use-case to segment 118 images covering Adelaide Island.We tested our approach by segmenting challenging historical images using a U-net model with just 80 training images, achieving an accuracy of 73% for 20 validation images. While no test data is available for our use case, a visual examination of the segmented images shows that our method performs effectively.The comparison of the hyper-parameters and augmentation methods provides directions for training other U-net-based models so that the presented workflow can be used to segment other archives with historical imagery. Additionally, the labelled training data and the segmented images of the test are publicly available at https://github.com/fdahle/antarctic_segmentation.

The ATLAS sensor onboard the ICESat-2 satellite is a photon-counting lidar (PCL) with a primary mission to map Earth's ice sheets. A secondary goal of the mission is to provide vegetation and terrain elevations, which are essential for calculating the planet's biomass carbon reserves. A drawback of ATLAS is that the sensor does not provide reliable terrain height estimates in dense, high-closure forests because only a few photons reach the ground through the canopy and return to the detector. This low penetration translates into lower accuracy for the resultant terrain model. Tropical forest measurements with ATLAS have an additional problem estimating top of canopy because of frequent atmospheric phenomena such as fog and low clouds that can be misinterpreted as top of the canopy. To alleviate these issues, we propose using a ConvPoint neural network for 3D point clouds and high-density airborne lidar as training data to classify vegetation and terrain returns from ATLAS. The semantic segmentation network provides excellent results and could be used in parallel with the current ATL08 noise filtering algorithms, especially in areas with dense vegetation. We use high-density airborne lidar data acquired along ICESat-2 transects in Central American forests as a ground reference for training the neural network to distinguish between noise photons and photons lying between the terrain and the top of the canopy. Each photon event receives a label (noise or signal) in the test phase, providing automated noise-filtering of the ATL03 data. The terrain and top of canopy elevations are subsequently aggregated in 100 m segments using a series of iterative smoothing filters. We demonstrate improved estimates for both terrain and top of canopy elevations compared to the ATL08 100 m segment estimates. The neural network (NN) noise filtering reliably eliminated outlier top of canopy estimates caused by low clouds, and aggregated root mean square error (RMSE) decreased from 7.7 m for ATL08 to 3.7 m for NN prediction (18 test profiles aggregated). For terrain elevations, RMSE decreased from 5.2 m for ATL08 to 3.3 m for the NN prediction, compared to airborne lidar reference profiles.

UAV-mounted sensors can be used to estimate crop biophysical traits, offering an alternative to traditional field scouting. However, the high temporal resolution offered by UAV platforms, critical for identifying small differences in crop conditions, is rarely exploited throughout the entire growing season. This limits growers' ability to obtain timely information for real-time interventions. New findings support that it is possible to parametrize an entire crop growth cycle under different conditions by accumulating sufficient data over time and using logistic growth models to highlight growth patterns. A step forward would be to model crop growth cycle at the plant-level in order to anticipate the optimal harvest dates in each plot or quickly identify growth problematics. Individual plant monitoring can be achieved by combining high spatial resolution images with accurate segmentation algorithms. The main objective of the study was therefore to develop and validate an integrated pipeline based on multidimensional data to extract predictive growth metrics for crop monitoring at the plant-level under various field conditions. The plant growth monitoring workflow was based on a three-step design ultimately leading to decision-making and reporting. Lettuce (Lactuca sativa L.) was chosen as a model plant due to its simple geometry, rapid growth and simple cultivation method. Treatments were composed of contrasting cover crops. Overall, correlation analysis showed that UAV-derived morphological metrics are reliable proxies for harvested biomass throughout the growing season, especially in later stages (Spearman's ρ > 0.9) and can be used as growth indicators. Therefore, Logistic Growth Curves (LGCs) were fitted to Crop Object Area (COA) values for each individual lettuce, using data up to 26 (generating G26 LGCs), 30 (G30) and 37 (G37) Days After Transplant (DAT). To assess the quality of their projections, G26 and G30 were compared to the reference LGC G37. The results indicated that Mean Absolute Percentage Error (MAPE) of projected COA was 9.6% and 6.8% for G26 and G30 respectively. Overall, the LGC parameters were close to the reference and highly correlated with the harvested biomass. The study also demonstrated the potential of having very good insight on plant maturity level by modeling the LGC 13 days before harvest. Furthermore, a dashboard was proposed to monitor current and projected maturity level, highlighting areas for further investigation. This novel integrated pipeline has the potential to become a valuable tool for research, on-farm decision making, and field interventions by providing data on plant biomass, maturity, and growth stages under different conditions, used as crop growth indicators.

The cover and management factor (C-factor) of the Universal Soil Loss Equation (USLE) represents the effects of crop cover, weighted by rainfall pattern, on predicted soil erosion rates. This requires an estimate of seasonal rainfall erosivity and soil protection afforded by the crop at different phenological stages, expressed by a soil loss ratio (SLR). However, soil erosion modelers often rely on vegetation-index-based regressions to directly estimate the cover and management factor (C-factor) of the USLE from satellite images. Since this approach is based on a single or very few images, it does not characterize the seasonality of the crop cover or reflect the seasonality of the rainfall erosivity. Here, we evaluated five vegetation indices (NDVI, NDRE, SFDVI, ViGREEN, and MGRVI) in predicting SLRs and the C-factor for a sugarcane plot in Southeast Brazil. We used Sentinel-2 images and orthomosaics obtained by UAV surveys performed at the middle of each phenological stage. We compared the estimates of the C-factor based on the SLRs and rainfall erosivity against direct regressions from the literature. Our results confirmed the expected poor correlation between the C-factor and the vegetation indices. On the other hand, using the proposed vegetation indices proved to be a reliable alternative to predict the SLR in sugarcane areas, especially the NDVI, the NDRE, and MGRVI. In particular, the MGRVI accurately predicted the SLR and classified the UAV-derived orthomosaics.

The compatibility of multispectral (MS) pansharpening algorithms with hyperspectral (HS) data is limited. With the recent development in HS satellites, there is a need for methods that can provide high spatial and spectral fidelity in both HS and MS scenarios.The present article presents a fast pansharpening method, based on the division of similar hyperspectral data in spectral subgroups using k-means clustering and Spectral Angle Mapper (SAM) profiling. Local Partial Least-Square (PLS) models are calibrated for each spectral subgroup against the respective pixels of the panchromatic image. The models are inverted to retrieve high-resolution pansharpened images. The method is tested against different methods that are able to handle both MS and HS pansharpening and assessed using reduced- and full-resolution evaluation methodologies. Based on a statistical multivariate approach, the proposed method is able to render uncertainty maps for spectral or spatial fidelity - functionality not reported in any other pansharpening study.

This paper presents Pol-InSAR-Island, the first publicly available multi-frequency Polarimetric Interferometric Synthetic Aperture Radar (Pol-InSAR) dataset labeled with detailed land cover classes, which serves as a challenging benchmark dataset for land cover classification. In recent years, machine learning has become a powerful tool for remote sensing image analysis. While there are numerous large-scale benchmark datasets for training and evaluating machine learning models for the analysis of optical data, the availability of labeled SAR or, more specifically, Pol-InSAR data is very limited. The lack of labeled data for training, as well as for testing and comparing different approaches, hinders the rapid development of machine learning algorithms for Pol-InSAR image analysis. The Pol-InSAR-Island benchmark dataset presented in this paper aims to fill this gap. The dataset consists of Pol-InSAR data acquired in S- and L-band by DLR's airborne F-SAR system over the East Frisian island Baltrum. The interferometric image pairs are the result of a repeat-pass measurement with a time offset of several minutes. The image data are given as 6 × 6 coherency matrices in ground range on a 1 m × 1m grid. Pixel-accurate class labels, consisting of 12 different land cover classes, are generated in a semi-automatic process based on an existing biotope type map and visual interpretation of SAR and optical images. Fixed training and test subsets are defined to ensure the comparability of different approaches trained and tested prospectively on the Pol-InSAR-Island dataset. In addition to the dataset, results of supervised Wishart and Random Forest classifiers that achieve mean Intersection-over-Union scores between 24% and 67% are provided to serve as a baseline for future work. The dataset is provided via KITopenData: https://doi.org/10.35097/1700.

The application of deep learning methods to remote sensing data has produced good results in recent studies. A promising application area is automatic land cover classification (semantic segmentation) from very high-resolution satellite imagery. However, the deep learning methods require large, labelled training datasets that are suitable for the study area. Map data can be used as training data, but it is often insufficiently detailed for very high-resolution satellite imagery. National airborne laser scanner (lidar) datasets provide additional details and are available in many countries. Successful land cover classifications from lidar datasets have been reached, e.g., by object-based image analysis. In the present study, we investigated the feasibility of using airborne laser scanner data and object-based image analysis to automatically generate labelled training data for a deep neural network -based land cover classification of a VHR satellite image. Input data for the object-based classification included digital surface models, intensity and pulse information derived from the lidar data. The resulting land cover classification was then utilized as training data for deep learning. A state-of-the-art deep learning architecture, UnetFormer, was trained and applied to the land cover classification of a WorldView-3 stereo dataset. For the semantic segmentation, three different input data composites were produced using the red, green, blue, NIR and digital surface model bands derived from the satellite data. The quality of the generated training data and the semantic segmentation results was estimated using an independent test set of ground truth points. The results show that final satellite image classification accuracy (94–96%) close to the training data accuracy (97%) was obtained. It was also demonstrated that the resulting maps could be used for land cover change detection.

Fine-grained information on the level of individual trees constitute key components for forest observation enabling forest management practices tackling the effects of climate change and the loss of biodiversity in forest ecosystems. Such information on individual tree crowns (ITC's) can be derived from the application of ITC segmentation approaches, which utilize remotely sensed data. However, many ITC segmentation approaches require prior knowledge about forest characteristics, which is difficult to obtain for parameterization. This can be avoided by the adoption of data-driven, automated workflows based on convolutional neural networks (CNN). To contribute to the advancements of efficient ITC segmentation approaches, we present a novel ITC segmentation approach based on the YOLOv5 CNN. We analyzed the performance of this approach on a comprehensive international unmanned aerial laser scanning (UAV-LS) dataset (ForInstance), which covers a wide range of forest types. The ForInstance dataset consists of 4192 individually annotated trees in high-density point clouds with point densities ranging from 498 to 9529 points m-2 collected across 80 sites. The original dataset was split into 70% for training and validation and 30% for model performance assessment (test data). For the best performing model, we observed a F1-score of 0.74 for ITC segmentation and a tree detection rate (DET %) of 64% in the test data. This model outperformed an ITC segmentation approach, which requires prior knowledge about forest characteristics, by 41% and 33% for F1-score and DET %, respectively. Furthermore, we tested the effects of reduced point densities (498, 50 and 10 points per m-2) on ITC segmentation performance. The YOLO model exhibited promising F1-scores of 0.69 and 0.62 even at point densities of 50 and 10 points m-2, respectively, which were between 27% and 34% better than the ITC approach that requires prior knowledge.Furthermore, the areas of ITC segments resulting from the application of the best performing YOLO model were close to the reference areas (RMSE = 3.19 m-2), suggesting that the YOLO-derived ITC segments can be used to derive information on ITC level.

The Unmanned aerial vehicles (UAVs) - based imaging is being intensively explored for precise crop evaluation. Various optical sensors, such as RGB, multi-spectral, and hyper-spectral cameras, can be used for this purpose. Consistent image quality is crucial for accurate plant trait prediction (i.e., phenotyping). However, achieving consistent image quality can pose a challenge as image qualities can be affected by i) UAV and camera technical settings, ii) environment, and iii) crop and field characters which are not always under the direct control of the UAV operator. Therefore, capturing the images requires the establishment of robust protocols to acquire images of suitable quality, and there is a lack of systematic studies on this topic in the public domain. Therefore, in this case study, we present an approach (protocols, tools, and analytics) that addressed this particular gap in our specific context. In our case, we had the drone (DJI Inspire 1 Raw) available, equipped with RGB camera (DJI Zenmuse x5), which needed to be standardized for phenotyping of the annual crops’ canopy cover (CC). To achieve this, we have taken 69 flights in Hyderabad, India, on 5 different cereal and legume crops (∼300 genotypes) in different vegetative growth stages with different combinations of technical setups of UAV and camera and across the environmental conditions typical for that region. For each crop-genotype combination, the ground truth (for CC) was rapidly estimated using an automated phenomic platform (LeasyScan phenomics platform, ICRISAT). This data-set enabled us to 1) quantify the sensitivity of image acquisition to the main technical, environmental and crop-related factors and this analysis was then used to develop the image acquisition protocols specific to our UAV-camera system. This process was significantly eased by automated ground-truth collection. We also 2) identified the important image quality indicators that integrated the effects of 1) and these indicators were used to develop the quality control protocols for inspecting the images post accquisition. To ease 2), we present a web-based application available at (https://github.com/GattuPriyanka/Framework-for-UAV-image-quality.git) which automatically calculates these key image quality indicators.Overall, we present a methodology for establishing the image acquisition protocol and quality check for obtained images, enabling a high accuracy of plant trait inference. This methodology was demonstrated on a particular UAV-camera set-up and focused on a specific crop trait (CC) at the ICRISAT research station (Hyderabad, India). We envision that, in the future, a similar image quality control system could facilitate the interoperability of data from various UAV-imaging set-ups.

Change detection from traditional 2D optical images has limited capability to model the changes in the height or shape of objects. Change detection using 3D point cloud from photogrammetry or LiDAR surveying can fill this gap by providing critical depth information. While most existing machine learning based 3D point cloud change detection methods are supervised, they severely depend on the availability of annotated training data, which is in practice a critical point. To circumnavigate this dependence, we propose an unsupervised 3D point cloud change detection method mainly based on self-supervised learning using deep clustering and contrastive learning. The proposed method also relies on an adaptation of deep change vector analysis to 3D point cloud via nearest point comparison. Experiments conducted on an aerial LiDAR survey dataset show that the proposed method obtains higher performance in comparison to the traditional unsupervised methods, with a gain of about 9% in mean accuracy (to reach more than 85%). Thus, it appears to be a relevant choice in scenario where prior knowledge (labels) is not ensured. The code will be made available at https://github.com/IdeGelis/torch-points3d-SSL-DCVA.

BIM is a powerful tool for the construction industry as well as for various other industries, so that its use has increased massively in recent years. Laser scanners are usually used for the measurement, which, in addition to the high acquisition costs, also cause problems on reflective surfaces. The use of photogrammetric techniques for BIM in industrial plants, on the other hand, is less widespread and less automated. CAD software (for point cloud evaluation) contains at best automated reconstruction algorithms for pipes. Fittings, flanges or elbows require a manual reconstruction. We present a method for automated processing of photogrammetric images for modeling pipelines in industrial plants. For this purpose we use instance segmentation and reconstruct the components of the pipeline directly based on the edges of the segmented objects in the images. Hardware costs can be kept low by using photogrammetry instead of laser scanning. Besides the autmatic extraction and reconstruction of pipes, we have also implemented this for elbows and flanges. For object recognition, we fine-tuned different instance segmentation models using our own training data, while also testing various data augmentation techniques. The average precision varies depending on the object type. The best results were achieved with Mask R–CNN. Here, the average precision was about 40%. The results of the automated reconstruction were examined with regard to the accuracy on a test object in the laboratory. The deviations from the reference geometry were in the range of a few millimeters and were comparable to manual reconstruction. In addition, further tests were carried out with images from a plant. Provided that the objects were correctly and completely recognized, a satisfactory reconstruction is possible with the help of our method.

Thermal mapping of buildings can be one approach to assess the insulation, which is important in regard to upgrade buildings to increase energy efficiency and for climate change adaptation. Personal laser scanning (PLS) is a fast and flexible option that has become increasingly popular to efficiently map building facades. However, some measurement systems do not include sufficient colorization of the point cloud. In order to detect, map and reference any damages to building facades, it is of great interest to transfer images from RGB and thermal infrared (TIR) cameras to the point cloud. This study aims to answer the research question if a flexible tool can be developed, which enable such measurements with high spatial resolution and flexibility. Therefore, an image-to-geometry registration approach for rendered point clouds is combined with a deep learning (DL)-based image feature matcher to estimate the camera pose of arbitrary images in relation to the geometry, i.e. the point cloud, to map color information. We developed a research design for multi-modal image matching to investigate the alignment of RGB and TIR camera images to a PLS point cloud with intensity information using calibrated and un-calibrated images. The accuracies of the estimated pose parameters reveal the best performance of the registration for pre-calibrated, i.e. undistorted, RGB camera images. The alignment of un-calibrated RGB and TIR camera images to a point cloud is possible if sufficient and well-distributed 2D-3D feature matches between image and point cloud are available. Our workflow enables the colorization of point clouds with high accuracy using images with very different radiometric characteristics and image resolutions. Only a rough approximation of the camera pose is required and hence the approach reliefs strict sensor synchronization requirements.

We showed that a mobile handheld laser scanner (HHLS) provides useful features concerning the wood quality-influencing external structures of trees. When linked with wood properties measured at a sawmill utilizing state-of-the-art X-ray scanners, these data enable the training of various wood quality models for use in targeting and planning future wood procurement. A total of 457 Norway spruce sample trees (Picea abies (L.) H. Karst.) from 13 spruce-dominated stands in southeastern Finland were used in the study. All test sites were recorded with a ZEB Horizon HHLS, and the sample trees were tracked to a sawmill and subjected to X-rays. Two branch extraction techniques were applied to the HHLS point clouds: 1) a method developed in this study that was based on the density-based spatial clustering of applications with noise (DBSCAN) and 2) segmentation-based quantitative structure model (treeQSM). On average, the treeQSM method detected 46% more branches per tree than the DBSCAN did. However, compared with the X-rayed references, some of the branches detected by the treeQSM may either be false positives or so small in size that the X-rays are unable to detect them as knots, as the method overestimated the whorl count by 19% when compared with the X-rays. On the other hand, the DBSCAN method only detected larger branches and showed a −11% bias in whorl count. Overall, the DBSCAN underestimated knot volumes within trees by 6%, while the treeQSM overestimated them by 25%. When we input the HHLS features into a Random Forest model, the knottiness variables measured at the sawmill were predicted with R2s of 0.47–0.64. The results were comparable with previous results derived with the static terrestrial laser scanners. The obtained stem branching data are relevant for predicting wood quality attributes but do not provide data that are directly comparable with the X-ray features. Future work should combine terrestrial point clouds with dense above-canopy point clouds to overcome the limitations related to vertical coverage.

Tree species characterise biodiversity, health, economic potential, and resilience of an ecosystem, for example. Tree species classification based on remote sensing data, however, is known to be a challenging task. In this paper, we study for the first time the feasibility of tree species classification using high-density point clouds collected with an airborne close-range multispectral laser scanning system – a technique that has previously proved to be capable of providing stem curve and volume accurately and rapidly for standing trees. To this end, we carried out laser scanning measurements from a helicopter on 53 forest sample plots, each with a size of 32 m × 32 m. The plots covered approximately 5500 trees in total, including Scots pine (Pinus sylvestris L.), Norway spruce (Picea abies (L.) H.Karst.), and deciduous trees such as Downy birch (Betula pubescens Ehrh.) and Silverbirch (Betula pendula Roth). The multispectral laser scanning system consisted of integrated Riegl VUX-1HA, miniVUX-3UAV, and VQ-840-G scanners (Riegl GmbH, Austria) operating at wavelengths of 1550 nm, 905 nm, and 532 nm, respectively. A new approach, layer-by-layer segmentation, was developed for individual tree detection and segmentation from the dense point cloud data. After individual tree segmentation, 249 features were computed for tree species classification, which was tested with approximately 3000 trees. The features described the point cloud geometry as well as single-channel and multi-channel reflectance metrics. Both feature selection and the tree species classification were conducted using the random forest method. Using the layer-by-layer segmentation algorithm, trees in the dominant and co-dominant categories were found with detection rates of 89.5% and 77.9%, respectively, whereas suppressed trees were detected with a success rate of 15.2%–42.3%, clearly improving upon the standard watershed segmentation. The overall accuracy of the tree species classification was 73.1% when using geometric features from the 1550 nm scanner data and 86.6% when combining the geometric features with reflectance information of the 1550 nm data. The use of multispectral reflectance and geometric features improved the overall classification accuracy up to 90.8%. Classification accuracies were as high as 92.7% and 93.7% for dominant and co-dominant trees, respectively.

Precise single tree delineation allows for a more reliable determination of essential parameters such as tree species, height and vitality. Methods of instance segmentation are powerful neural networks for detecting and segmenting single objects and have the potential to push the accuracy of tree segmentation methods to a new level. In this study, two instance segmentation methods, Mask R–CNN and DETR, were applied to precisely delineate single tree crowns using multispectral images and images generated from UAV lidar data. The study area was in Bavaria, 35 km north of Munich (Germany), comprising a mixed forest stand of around 7 ha characterised mainly by Norway spruce (Picea abies) and large groups of European beeches (Fagus sylvatica) with 181–236 trees per ha. The data set, consisting of multispectral images and lidar data, was acquired using a Micasense RedEdge-MX dual camera system and a Riegl miniVUX-1UAV lidar scanner, both mounted on a hexacopter (DJI Matrice 600 Pro). At an altitude of approximately 85 m, two flight missions were conducted at an airspeed of 5 m/s, leading to a ground resolution of 5 cm and a lidar point density of 560 points/m2. In total, 1408 trees were marked by visual interpretation of the remote sensing data for training and validating the classifiers. Additionally, 125 trees were surveyed by tacheometric means used to test the optimized neural networks. The evaluations showed that segmentation using only multispectral imagery performed slightly better than with images generated from lidar data. In terms of F1 score, Mask R–CNN with color infrared (CIR) images achieved 92% in coniferous, 85% in deciduous and 83% in mixed stands. Compared to the images generated by lidar data, these scores are the same for coniferous and slightly worse for deciduous and mixed plots, by 4% and 2%, respectively. DETR with CIR images achieved 90% in coniferous, 81% in deciduous and 84% in mixed stands. These scores were 2%, 1%, and 2% worse, respectively, compared to the lidar data images in the same test areas. Interestingly, four conventional segmentation methods performed significantly worse than CIR-based and lidar-based instance segmentations. Additionally, the results revealed that tree crowns were more accurately segmented by instance segmentation. All in all, the results highlight the practical potential of the two deep learning-based tree segmentation methods, especially in comparison to baseline methods.

Forest characteristics vary largely at the regional level and in smaller geographic areas in Finland. The amount of greenhouse gas emissions is related to changes in biomass and the soil type (e.g. upland soils vs. peatlands). In this paper, estimating and explaining spatial patterns of tree biomass change across Finland was the main interest. We analysed biomass changes on different soil and site types between the years 2009 and 2015 using the Finnish multi-source national forest inventory (MS-NFI) raster layers. MS-NFI method is based on combining information from satellite imagery, digital maps and national forest inventory (NFI) field data. Automatic segmentation was used to create silvicultural management and treatment units. An average biomass estimate of the segmented MS-NFI (MS–NFI–seg) map was 73.9 tons ha−1 compared to the national forest inventory estimate of 76.5 tons ha−1 in 2015. Forest soil type had a similar effect on average biomass in MS–NFI–seg and NFI data. Despite good regional and country-level results, segmentation narrowed the biomass distributions. Hence, biomass changes on segments can be considered only approximate values; also, those small differences in average biomass may accumulate when map layers from more than one time point are compared. A kappa of 0.44 was achieved for precision when comparing undisturbed and disturbed forest stands in the segmented Global Forest Change data (GFC-seg) and MS–NFI–seg map. Compared to NFI, 69% and 62% of disturbed areas were detected by GFC-seg and MS–NFI–seg, respectively. Spatially accurate map data of biomass changes on forest land improve the ability to suggest optimal management alternatives for any patch of land, e.g. in terms of climate change mitigation.

First order geometric network design is an important quality assurance process for terrestrial laser scanning of complex built environments for the construction of digital as-built models. A key design task is the determination of a set of instrument locations or viewpoints that provide complete site coverage while meeting quality criteria. Although simplified point precision measures are often used in this regard, precision measures for common geometric objects found in the built environment—planes, cylinders and spheres—are arguably more relevant indicators of as-built model quality. The computation of such measures at the design stage—which is not currently done—requires generation of artificial observations by ray casting, which can be a dissuasive factor for their adoption. This paper presents models for the rigorous computation of geometric object precision without the need for ray casting. Instead, a model for the 2D distribution of angular observations is coupled with candidate viewpoint-object geometry to derive the covariance matrix of parameters. Three-dimensional models are developed and tested for vertical cylinders, spheres and vertical, horizontal and tilted planes. Precision estimates from real experimental data were used as the reference for assessing the accuracy of the predicted precision—specifically the standard deviation—of the parameters of these objects. Results show that the mean accuracy of the model-predicted precision was 4.3% (of the read data value) or better for the planes, regardless of plane orientation. The mean accuracy of the cylinders was up to 6.2%. Larger differences were found for some datasets due to incomplete object coverage with the reference data, not due to the model. Mean precision for the spheres was similar, up to 6.1%, following adoption of a new model for deriving the angular scanning limits. The computational advantage of the proposed method over precision estimates from simulated, high-resolution point clouds is also demonstrated. The CPU time required to estimate precision can be reduced by up to three orders of magnitude. These results demonstrate the utility of the derived models for efficiently determining object precision in 3D network design in support of scanning surveys for reality capture.

Increasing tree mortality due to climate change has been observed globally. Remote sensing is a suitable means for detecting tree mortality and has been proven effective for the assessment of abrupt and large-scale stand-replacing disturbances, such as those caused by windthrow, clear-cut harvesting, or wildfire. Non-stand replacing tree mortality events (e.g., due to drought) are more difficult to detect with satellite data – especially across regions and forest types. A common limitation for this is the availability of spatially explicit reference data. To address this issue, we propose an automated generation of reference data using uncrewed aerial vehicles (UAV) and deep learning-based pattern recognition. In this study, we used convolutional neural networks (CNN) to semantically segment crowns of standing dead trees from 176 UAV-based very high-resolution (

Protected agriculture boosts the production of vegetables, berries and fruits, and it plays a pivotal role in guaranteeing food security globally in the face of climate change. Remote sensing is proven to be useful for identifying the presence of (low-tech) plastic greenhouses and plastic mulches. However, the classification accuracy notoriously decreases in the presence of small-scale farming, heterogeneous land cover and unaccounted seasonal management of protected agriculture. Here, we present the random forest-based pixel-level Open field and Protected Agriculture land cover Classifier (OPAC) developed using Sentinel-2 L2A data. OPAC is trained using tiles from Switzerland over 2 years and the Almeria region in Spain over 1 acquisition day. OPAC classifies eight land covers typical of open field and protected agriculture (plastic mulches, low-tech greenhouses and for the first time high-tech greenhouses). Finally, we assess (1) how the land covers in OPAC are labelled in the Sentinel-2 Scene Classification Layer (SCL) and (2) the correspondence between pixels classified as protected agriculture by OPAC and by the best performing Advanced Plastic Greenhouse Index (APGI). To reduce anthropogenic land covers, we constrain the classification task to agricultural areas retrieved from cadastral data or the Corine Land Cover map. The 5-fold cross-validation reveals an overall accuracy of 92% but other classification scores are moderate when keeping the separation among the three classes of protected agriculture. However, all scores substantially improve upon grouping the three classes into one (with an Intersection Over Union of 0.58 as an average among the scores of the three classes and of 0.98 for one single class). Given the recently acknowledged importance of Sentinel-2 Band 1 (central wavelength of 443 nm), the classification accuracy of OPAC for the Swiss small-scale farming is mostly limited by the band's reduced spatial accuracy (60 m). A careful visual assessment indicates that OPAC achieves satisfactory generalization capabilities also in North European (the Netherlands) and four Mediterranean areas (Spain, Italy, Crete and Turkey) without the need of adding location and temporal specific information. There is good agreement among natural land covers classified by OPAC and the SCL. However, the SCL does not have a class for protected agriculture, the latter being often classified as clouds. APGI achieved similar to lower classification accuracies than OPAC. Importantly, the APGI classification task depends on a user-defined space- and time-specific threshold, whereas OPAC does not. Therefore, OPAC paves the way for rapid mapping of protected agriculture at continental scale.

Three-dimensional (3D) point cloud registration is a fundamental step for many 3D modeling and mapping applications. Existing approaches are highly disparate in the data source, scene complexity, and application, therefore the current practices in various point cloud registration tasks are still ad-hoc processes. Recent advances in computer vision and deep learning have shown promising performance in estimating rigid/similarity transformation between unregistered point clouds of complex objects and scenes. However, their performances are mostly evaluated using a limited number of datasets from a single sensor (e.g. Kinect or RealSense cameras), lacking a comprehensive overview of their applicability in photogrammetric 3D mapping scenarios. In this work, we provide a comprehensive review of the state-of-the-art (SOTA) point cloud registration methods, where we analyze and evaluate these methods using a diverse set of point cloud data from indoor to satellite sources. The quantitative analysis allows for exploring the strengths, applicability, challenges, and future trends of these methods. In contrast to existing analysis works that introduce point cloud registration as a holistic process, our experimental analysis is based on its inherent two-step process to better comprehend these approaches including feature/keypoint-based initial coarse registration and dense fine registration through cloud-to-cloud (C2C) optimization. More than ten methods, including classic hand-crafted, deep-learning-based feature correspondence, and robust C2C methods were tested. We observed that the success rate of most of the algorithms are fewer than 40% over the datasets we tested and there are still are large margin of improvement upon existing algorithms concerning 3D sparse corresopondence search, and the ability to register point clouds with complex geometry and occlusions. With the evaluated statistics on three datasets, we conclude the best-performing methods for each step and provide our recommendations, and outlook future efforts.

Land Use Land Cover (LULC) segmentation is a famous application of remote sensing in an urban environment. Up-to-date and complete data are of major importance in this field. Although with some success, pixel-based segmentation remains challenging because of class variability. Due to the increasing popularity of crowd-sourcing projects, like OpenStreetMap, the need for user-generated content has also increased, providing a new prospect for LULC segmentation. We propose a deep-learning approach to segment objects in high-resolution imagery by using semantic crowdsource information. Due to satellite imagery and crowdsource database complexity, deep learning frameworks perform a significant role. This integration reduces computation and labor costs. Our methods are based on a fully convolutional neural network (CNN) that has been adapted for multi-source data processing. We discuss the use of data augmentation techniques and improvements to the training pipeline. We applied semantic (U-Net) and instance segmentation (Mask R-CNN) methods and, Mask R–CNN showed a significantly higher segmentation accuracy from both qualitative and quantitative viewpoints. The conducted methods reach 91% and 96% overall accuracy in building segmentation and 90% in road segmentation, demonstrating OSM and remote sensing complementarity and potential for city sensing applications.

This paper proposes a novel method to improve georeferencing of airborne laser scanning by improved trajectory estimation using Vehicle Dynamic Model. In Vehicle Dynamic Model (VDM), the relationship between the dynamics of the platform and control inputs is used as additional observations for sensor fusion. This relationship is available for most platforms and can be used without the need for additional hardware. However, this relationship is modeled using parameters that are a priori unknown. The proposed in-flight calibration methodology can achieve less than 2% error in the estimated model parameters compared to the values used in simulation. The effect of Inertial Measurement Unit (IMU) noise on the accuracy of airborne laser scanning is further investigated to demonstrate the reduction in the position error of georeferenced points when VDM measurements are used. The results are evaluated through a Monte-Carlo simulation involving an open-source autopilot. The reduction in the error of the estimated attitude due to vehicle modeling increases with the higher intensity of time-correlated IMU noise. Using a higher quality inertial sensor does not lead to an improvement in the position error of georeferenced points when VDM measurements are employed; however, a lower quality Inertial Measurement Unit, such as those on an autopilot, shows a 33% and 46% reduction in the mean and standard deviation of the position error of the georeferenced points, respectively.

Light detection and ranging (lidar) scanning systems can be used to provide a point cloud with high quality and point density. Gridded digital elevation models (DEMs) interpolated from laser scanning point clouds are widely used due to their convenience, however, DEM uncertainty is rarely provided. This paper proposes an end-to-end workflow to quantify the uncertainty (i.e., standard deviation) of a gridded lidar-derived DEM. A benefit of the proposed approach is that it does not require independent validation data measured by alternative means. The input point cloud requires per point uncertainty which is derived from lidar system observational uncertainty. The propagated uncertainty caused by interpolation is then derived by the general law of propagation of variances (GLOPOV) with simultaneous consideration of both horizontal and vertical point cloud uncertainties. Finally, the interpolated uncertainty is then scaled by point density and a measure of terrain roughness to arrive at the final gridded DEM uncertainty. The proposed approach is tested with two lidar datasets measured in Waikoloa, Hawaii, and Sitka, Alaska. Triangulated irregular network (TIN) interpolation is chosen as the representative gridding approach. The results indicate estimated terrain roughness/point density scale factors ranging between 1 (in flat areas) and 7.6 (in high roughness areas), with a mean value of 2.3 for the Waikoloa dataset and between 1 and 9.2 with a mean value of 1.2 for the Sitka dataset. As a result, the final gridded DEM uncertainties are estimated between 0.059 m and 0.677 m with a mean value of 0.164 m for the Waikoloa dataset and between 0.059 m and 1.723 m with a mean value of 0.097 m for the Sitka dataset.

Among photogrammetric products, orthophotos are probably the most versatile and widely used in many fields of application. In the last years, coupled with the spread of semi-automated survey and processing approaches based on photogrammetry, orthophotos have become almost a standard for monitoring the underwater environment. If on land the definition of the reference coordinate system and projection plane for the orthophoto generation is trivial, underwater it may represent a challenge. In this paper, we address the issue of defining the vertical direction and resulting horizontal plane (levelling) for the differential ortho rectification. We propose a non-invasive, contactless method based on roll and pitch angular data provided by in-camera IMU sensors and embedded in the Exif metadata of JPEG and raw image files. We show how our approach can be seamlessly integrated into automatic SfM/MVS pipelines, provide the mathematical background, and showcase real-world applications results in an underwater monitoring project. The results illustrate the effectiveness of the proposed method and, for the first time, provide a metric evaluation of the definition of the vertical direction with low-cost sensors enclosed in digital cameras directly underwater.

Historical terrestrial images are the only visual sources documenting alpine environments shortly after the end of the Little Ice Age. Despite their unique value, they are largely unused for quantifying environmental changes because of the difficult and time-consuming estimation of the unknown camera parameters. For most images large parts of the captured scenery have vastly changed over time, making automatic feature point matching infeasible. In contrast, the visible image horizon seems to remain stable over time and hence, appears to be a suitable feature for image orientation. Since the focal length is unknown for historical terrestrial images, existing methods, focusing solely on estimating the exterior orientation of recent imagery, can not be applied. Accordingly, it was investigated if the horizon is suitable to estimate both the interior and exterior orientation of historical terrestrial images, with an accuracy comparable to manually oriented images. In a first step, the whole horizon was used to approximate the unknown camera parameters, reducing the potential search space. In the subsequent spatial resection these approximations were further refined using salient points along the horizon. We evaluated our approach using 204 manually oriented reference images. With the proposed method the accuracy of the estimated exterior orientation could be significantly improved compared to previous works. Additionally, the unknown focal length was estimated within 5% of the true focal length for 75% of the images. As historical terrestrial images are commonly used for monoplotting, the accuracy for 2400 manually selected checkpoints was evaluated. This analysis showed that for 63% of the images the same accuracy as with manually oriented images was achieved. For additional 22% the estimated camera parameters were still accurate enough to serve as initial estimates for a subsequent manual orientation. In 15% of the images our method completely failed. Due to the vastly changing scenery and oblique viewing geometry, finding the initial camera parameters, in our experience, is often the most challenging and time consuming step during manual orientation of historical images. Hence, in 85% of the images this initial step can be replaced with our method, leading to a significantly reduced effort for orienting whole collections of historical terrestrial images.

Mapping standing dead trees, especially, in natural forests is very important for evaluation of the forest's health status, and its capability for storing Carbon, and the conservation of biodiversity. Apparently, natural forests have larger areas which renders the classical field surveying method very challenging, time-consuming, labor-intensive, and unsustainable. Thus, for effective forest management, there is the need for an automated approach that would be cost-effective. With the advent of Machine Learning, Deep Learning has proven to successfully achieve excellent results. This study presents an adjusted Mask R-CNN Deep Learning approach for detecting and segmenting standing dead trees in a mixed dense forest from CIR aerial imagery using a limited (195 images) training dataset. First, transfer learning is considered coupled with the image augmentation technique to leverage the limitation of training datasets. Then, we strategically selected hyperparameters to suit appropriately our model's architecture that fits well with our type of data (dead trees in images). Finally, to assess the generalization capability of our model's performance, a test dataset that was not confronted to the deep neural network was used for comprehensive evaluation. Our model recorded promising results reaching a mean average precision, average recall, and average F1-Score of 0.85, 0.88, and 0.87 respectively, despite our relatively low resolution (20 cm) dataset. Consequently, our model could be used for automation in standing dead tree detection and segmentation for enhanced forest management. This is equally significant for biodiversity conservation, and forest Carbon storage estimation.

Unoccupied aerial vehicles (UAV) with RGB-cameras are affordable and versatile devices for the generation of a series of remote sensing products that can be used for forest inventory tasks, such as creating high-resolution orthomosaics and canopy height models. The latter may serve purposes including tree species identification, forest damage assessments, canopy height or timber stock assessments. Besides flight and image acquisition parameters such as image overlap, flight height, and weather conditions, the focal length, which determines the opening angle of the camera lens, is a parameter that influences the reconstruction quality. Despite its importance, the effect of focal length on the quality of 3D reconstructions of forests has received little attention in the literature. Shorter focal lengths result in more accurate distance estimates in the nadir direction since small angular errors lead to large positional errors in narrow opening angles. In this study, 3D reconstructions of four UAV-acquisitions with different focal lengths (21, 35, 50, and 85 mm) on a 1 ha mature mixed forest plot were compared to reference point clouds derived from high quality Terrestrial Laser Scans. Shorter focal lengths (21 and 35 mm) led to a higher agreement with the TLS scans and thus better reconstruction quality, while at 50 mm, quality losses were observed, and at 85 mm, the quality was considerably worse. F1-scores calculated from a voxel representation of the point clouds amounted to 0.254 with 35 mm and 0.201 with 85 mm. The precision with 21 mm focal length was 0.466 and 0.302 with 85 mm. We thus recommend a focal length no longer than 35 mm during UAV Structure from Motion (SfM) data acquisition for forest management practices.

The increased availability of satellite SAR data coupled with improved InSAR processing algorithms has led to higher accuracy of InSAR-derived displacements. However, obtaining millimeter-level accuracy still remains a challenge due to the inability to accurately model the tropospheric delay, which is sometimes larger than the actual deformation. We propose to estimate the relative daily SAR tropospheric delay by double differencing interferograms that share a common image by leveraging Sentinel-1’s short revisit time and the interchanging role of the common image. The resulting double-differenced interferograms consist of tropospheric delay and processing noise. This combination leads to an inverse problem as a result of the differential nature of the interferograms. The unknown SAR tropopsheric delay is solved by Tikhonov regularization and the processing errors are accounted for by covariance modelling. In cases of rapid localized displacement, the deforming region is masked and the tropospheric delay for the deforming region is determined through kriging. Validation is performed using simulated SAR data and GNSS tropospheric delays. We find a good correlation between estimated SAR tropospheric delays and GNSS tropospheric delays, with an average root mean square error (RMSE) of 1.93 radians across six GNSS locations. In addition, we examine the performance of the correction on interferograms through variogram modelling, which indicates improved correction both for short-wavelength and long-wavelength tropospheric noise. SAR tropospheric delay estimates are integrated into a multitemporal InSAR workflow through either interferometric subtraction or stochastic modelling. Validation using GNSS measurements indicates that SAR tropospheric delay estimates significantly improve the accuracy of the InSAR-derived time-series displacement.