Search

Precise and accurate three-dimensional geospatial data has become increasingly available thanks to advances in both Terrestrial Laser Scanning (TLS) and Structure-from-Motion Photogrammetry (SfM). These tools provide valuable information for mapping geomorphological features and detect surface changes in mountainous environments. The exploitation of 3D point-clouds has been proven tremendously useful in the field of geosciences. It remains, however, controversial whether cost efficient photogrammetry can provide as accurate and reliable geospatial information as the significantly more expensive laser scanning or not. In this study, a rockfall case site in the territory of Obergurgl, Austria, is investigated in order to provide answers to the above question in a complex environment. The analysis includes different terrestrial photogrammetry configurations aiming to comprehensively define the strengths and limitations of terrestrial photogrammetry over TLS. The latter constitutes an optimized methodology that provides guidelines for costly future assessments as part of the site investigation phase in geohazard management. There are no doubts that compared to traditional and conventional surveying methods TLS and Photogrammetry both offer products much faster and with a much higher data density. In the current study, we show that when photogrammetry is applied following a well-defined optimized strategy, it can be potentially an adequate alternative to more costly TLS datasets for mass movement assessment and monitoring purposes.

Deep learning methods based on Fully convolution networks (FCNs) have shown an impressive progress in building outline delineation from very high resolution (VHR) remote sensing (RS) imagery. Common issues still exist in extracting precise building shapes and outlines, often resulting in irregular edges and over smoothed corners. In this paper, we use PolyMapper, a recently introduced deep-learning framework that is able to predict object outlines in a vector representation directly. We have introduced two main modifications to this baseline method. First, we introduce EffcientNet as backbone feature encoder to our network, which uses compound coefficient to scale up all dimensions of depth/width/resolution uniformly, to improve the processing speed with fewer parameters. Second, we integrate a boundary refinement block (BRB) to strengthen the boundary feature learning and to further improve the accuracy of corner prediction. The results demonstrate that the end-to-end learnable model is capable of delineating polygons of building outlines that closely approximate the structure of reference labels. Experiments on the crowdAI building instance segmentation datasets show that our model outperforms PolyMapper in all COCO metrics, for instance showing a 0.13 higher mean Average Precision (AP) value and a 0.60 higher mean Average Recall value. Also qualitative results show that our method segments building instances of various shapes more accurately.

Semantic segmentation models are often affected by illumination changes, and fail to predict correct labels. Although there has been a lot of research on indoor semantic segmentation, it has not been studied in low-light environments. In this paper we propose a new framework, LISU, for Low-light Indoor Scene Understanding. We first decompose the low-light images into reflectance and illumination components, and then jointly learn reflectance restoration and semantic segmentation. To train and evaluate the proposed framework, we propose a new data set, namely LLRGBD, which consists of a large synthetic low-light indoor data set (LLRGBD-synthetic) and a small real data set (LLRGBD-real). The experimental results show that the illumination-invariant features effectively improve the performance of semantic segmentation. Compared with the baseline model, the mIoU of the proposed LISU framework has increased by 11.5%. In addition, pre-training on our synthetic data set increases the mIoU by 7.2%. Our data sets and models are available on our project website.

Deep learning-based semantic segmentation models for building delineation face the challenge of producing precise and regular building outlines. Recently, a building delineation method based on frame field learning was proposed by Girard et al. (2020) to extract regular building footprints as vector polygons directly from aerial RGB images. A fully convolution network (FCN) is trained to learn simultaneously the building mask, contours, and frame field followed by a polygonization method. With the direction information of the building contours stored in the frame field, the polygonization algorithm produces regular outlines accurately detecting edges and corners. This paper investigated the contribution of elevation data from the normalized digital surface model (nDSM) to extract accurate and regular building polygons. The 3D information provided by the nDSM overcomes the aerial images’ limitations and contributes to distinguishing the buildings from the background more accurately. Experiments conducted in Enschede, the Netherlands, demonstrate that the nDSM improves building outlines’ accuracy, resulting in better-aligned building polygons and prevents false positives. The investigated deep learning approach (fusing RGB + nDSM) results in a mean intersection over union (IOU) of 0.70 in the urban area. The baseline method (using RGB only) results in an IOU of 0.58 in the same area. A qualitative analysis of the results shows that the investigated model predicts more precise and regular polygons for large and complex structures.

Modern machine learning, especially deep learning, which is used in a variety of applications, requires a lot of labelled data for model training. Having an insufficient amount of training examples leads to models which do not generalize well to new input instances. This is a particular significant problem for tasks involving aerial images: often training data is only available for a limited geographical area and a narrow time window, thus leading to models which perform poorly in different regions, at different times of day, or during different seasons. Domain adaptation can mitigate this issue by using labelled source domain training examples and unlabeled target domain images to train a model which performs well on both domains. Modern adversarial domain adaptation approaches use unpaired data. We propose using pairs of semantically similar images, i.e., whose segmentations are accurate predictions of each other, for improved model performance. In this paper we show that, as an upper limit based on ground truth, using semantically paired aerial images during training almost always increases model performance with an average improvement of 4.2% accuracy and .036 mean intersection-over-union (mIoU). Using a practical estimate of semantic similarity, we still achieve improvements in more than half of all cases, with average improvements of 2.5% accuracy and .017 mIoU in those cases.

Land use (LU) is an important information source commonly stored in geospatial databases. Most current work on automatic LU classification for updating topographic databases considers only one category level (e.g. residential or agricultural) consisting of a small number of classes. However, LU databases frequently contain very detailed information, using a hierarchical object catalogue where the number of categories differs depending on the hierarchy level. This paper presents a method for the classification of LU on the basis of aerial images that differentiates a fine-grained class structure, exploiting the hierarchical relationship between categories at different levels of the class catalogue. Starting from a convolutional neural network (CNN) for classifying the categories of all levels, we propose a strategy to simultaneously learn the semantic dependencies between different category levels explicitly. The input to the CNN consists of aerial images and derived data as well as land cover information derived from semantic segmentation. Its output is the class scores at three different semantic levels, based on which predictions that are consistent with the class hierarchy are made. We evaluate our method using two test sites and show how the classification accuracy depends on the semantic category level. While at the coarsest level, an overall accuracy in the order of 90% can be achieved, at the finest level, this accuracy is reduced to around 65%. Our experiments also show which classes are particularly hard to differentiate. © 2020 Copernicus GmbH. All rights reserved.

In recent years, the determination of global image orientation, i.e. global SfM, has gained a lot of attentions from researchers, mainly due to its time efficiency. Most of the global methods take relative rotations and translations as input for a two-step strategy comprised of global rotation averaging and global translation averaging. This paper by contrast presents a hybrid approach that aims to solve global rotations and translations simultaneously, but hierarchically. We first extract an optimal minimum cover connected image triplet set (OMCTS) which includes all available images with a minimum number of triplets, all of them with the three related relative orientations being compatible to each other. For non-collinear triplets in the OMCTS, we introduce some basic characterizations of the corresponding essential matrices and solve for the image pose parameters by averaging the constrained essential matrices. For the collinear triplets, on the other hand, the image pose parameters are estimated by relative orientation using the depth of object points from individual local spatial intersection. Finally, all image orientations are estimated in a common coordinate frame by traversing every solved triplet using a similarity transformation. We show results of our method on different benchmarks and demonstrate the performance and capability of the proposed approach by comparing with other global SfM methods.

Many countries were the target of air strikes during World War II. Numerous unexploded bombs still exist in the ground. These duds can be tracked down with the help of bomb craters, indicating areas where unexploded bombs may be located. Such areas are documented in so-called impact maps based on detected bomb craters. In this paper, a stochastic approach based on marked point processes (MPPs) for the automatic detection of bomb craters in aerial images taken during World War II is presented. As most areas are covered by multiple images, the influence of redundant image information on the object detection result is investigated: we compare the results generated based on single images with those obtained by our new approach that combines the individual detection results of multiple images covering the same location. The object model for the bomb craters is represented by circles. Our MPP approach determines the most likely configuration of objects within the scene. The goal is reached by minimizing an energy function that describes the conformity with a predefined model by Reversible Jump Markov Chain Monte Carlo sampling in combination with simulated annealing. Afterwards, a probability map is generated from the automatic detections via kernel density estimation. By setting a threshold, areas around the detections are classified as contaminated or uncontaminated sites, respectively, which results in an impact map. Our results show a significant improvement with respect to its quality when redundant image information is used.

Fully convolutional neural networks (FCN) are successfully used for the automated pixel-wise classification of aerial images and possibly additional data. However, they require many labelled training samples to perform well. One approach addressing this issue is semi-supervised domain adaptation (SSDA). Here, labelled training samples from a source domain and unlabelled samples from a target domain are used jointly to obtain a target domain classifier, without requiring any labelled samples from the target domain. In this paper, a two-step approach for SSDA is proposed. The first step corresponds to a supervised training on the source domain, making use of strong data augmentation to increase the initial performance on the target domain. Secondly, the model is adapted by entropy minimization using a novel weighting strategy. The approach is evaluated on the basis of five domains, corresponding to five cities. Several training variants and adaptation scenarios are tested, indicating that proper data augmentation can already improve the initial target domain performance significantly resulting in an average overall accuracy of 77.5%. The weighted entropy minimization improves the overall accuracy on the target domains in 19 out of 20 scenarios on average by 1.8%. In all experiments a novel FCN architecture is used that yields results comparable to those of the best-performing models on the ISPRS labelling challenge while having an order of magnitude fewer parameters than commonly used FCNs.

This paper proposes several methods for training a Convolutional Neural Network (CNN) for learning the similarity between images of silk fabrics based on multiple semantic properties of the fabrics. In the context of the EU H2020 project SILKNOW (http://silknow.eu/), two variants of training were developed, one based on a Siamese CNN and one based on a triplet architecture. We propose different definitions of similarity and different loss functions for both training strategies, some of them also allowing the use of incomplete information about the training data. We assess the quality of the trained model by using the learned image features in a k-NN classification. We achieve overall accuracies of 93–95% and average F1-scores of 87–92%.

State-of-the-art object detection approaches such as Fast/Faster R-CNN, SSD, or YOLO have difficulties detecting dense, small targets with arbitrary orientation in large aerial images. The main reason is that using interpolation to align RoI features can result in a lack of accuracy or even loss of location information. We present the Local-aware Region Convolutional Neural Network (LR-CNN), a novel two-stage approach for vehicle detection in aerial imagery. We enhance translation invariance to detect dense vehicles and address the boundary quantization issue amongst dense vehicles by aggregating the high-precision RoIs' features. Moreover, we resample high-level semantic pooled features, making them regain location information from the features of a shallower convolutional block. This strengthens the local feature invariance for the resampled features and enables detecting vehicles in an arbitrary orientation. The local feature invariance enhances the learning ability of the focal loss function, and the focal loss further helps to focus on the hard examples. Taken together, our method better addresses the challenges of aerial imagery. We evaluate our approach on several challenging datasets (VEDAI, DOTA), demonstrating a significant improvement over state-of-the-art methods. We demonstrate the good generalization ability of our approach on the DLR 3K dataset., Comment: 8 pages

In this paper we present a novel approach for image orientation by combining relative rotations and tie points. First, we choose an initial image pair with enough correspondences and large triangulation angle, and we then iteratively add clusters of new images. The rotation of these newly added images is estimated from relative rotations by single rotation averaging. In the next step, a linear equation system is set up for each new image to solve the translation parameters with triangulated tie points which can be viewed in that new image, followed by a resection for refinement. Finally, we optimize the cluster of reconstructed images by local bundle adjustment. We show results of our approach on different benchmark datasets. Furthermore, we orient several larger datasets incl. unordered image datasets to demonstrate the robustness and performance of our approach.

In this paper we deal with the problem of measuring the similarity between training and tests datasets in the context of transfer learning (TL) for image classification. TL tries to transfer knowledge from a source domain, where labelled training samples are abundant but the data may follow a different distribution, to a target domain, where labelled training samples are scarce or even unavailable, assuming that the domains are related. Thus, the requirements w.r.t. the availability of labelled training samples in the target domain are reduced. In particular, if no labelled target data are available, it is inherently difficult to find a robust measure of relatedness between the source and target domains. This is of crucial importance for the performance of TL, because the knowledge transfer between unrelated data may lead to negative transfer, i.e. to a decrease of classification performance after transfer. We address the problem of measuring the relatedness between source and target datasets and investigate three different strategies to predict and, consequently, to avoid negative transfer in this paper. The first strategy is based on circular validation. The second strategy relies on the Maximum Mean Discrepancy (MMD) similarity metric, whereas the third one is an extension of MMD which incorporates the knowledge about the class labels in the source domain. Our method is evaluated using two different benchmark datasets. The experiments highlight the strengths and weaknesses of the investigated methods. We also show that it is possible to reduce the amount of negative transfer using these strategies for a TL method and to generate a consistent performance improvement over the whole dataset.

Classification and segmentation of buildings from airborne lidar point clouds commonly involve point features calculated within a local neighborhood. The relative change of the features in the immediate surrounding of each point as well as the spatial relationships between neighboring points also need to be examined to account for spatial coherence. In this study we formulate the point labeling problem under a global graph-cut optimization solution. We construct the energy function through a graph representing a Markov Random Field (MRF). The solution to the labeling problem is obtained by finding the minimum-cut on this graph. We have employed this framework for three different labeling tasks on airborne lidar point clouds. Ground filtering, building classification, and roof-plane segmentation. As a follow-up study on our previous ground filtering work, this paper examines our building extraction approach on two airborne lidar datasets with different point densities containing approximately 930K points in one dataset and 750K points in the other. Test results for building vs. non-building point labeling show a 97.9% overall accuracy with a kappa value of 0.91 for the dataset with 1.18 pts/m2 average point density and a 96.8% accuracy with a kappa value of 0.90 for the dataset with 8.83 pts/m2 average point density. We can achieve 91.2% overall average accuracy in roof plane segmentation with respect to the reference segmentation of 20 building roofs involving 74 individual roof planes. In summary, the presented framework can successfully label points in airborne lidar point clouds with different characteristics for all three labeling problems we have introduced. It is robust to noise in the calculated features due to the use of global optimization. Furthermore, the framework achieves these results with a small training sample size. © 2020 Copernicus GmbH. All rights reserved., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, V, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

Traditional flow velocity measurements in natural environments require contact with the fluid and are usually costly, time-consuming and, sometimes, even dangerous. Particle Image Velocimetry allows the flow velocity field to be remotely characterized from the shift of intensity patterns of sub-image areas in at least two video frames with a known time lag. Recently, Airborne Image Velocimetry has enabled the surface velocity field of large-scale water bodies to be determined by applying Particle Image Velocimetry on videos recorded by cameras mounted on unmanned aerial vehicles. This work presents a comparison of three Airborne Image Velocimetry approaches: BASESURV, Fudaa-LSPIV and RIVeR. For the evaluation, two nadiral videos were acquired with a low-cost quadcopter. The first was recorded under low flow and seeded conditions, the second during a flood event. According to the results obtained, BASESURV is an accurate and complete research oriented approach but it is time-consuming and neither a graphical interface nor documentation are yet provided. Fudaa-LSPIV is a well-developed software package, with a user-friendly graphical interface and good documentation. However it lacks some features and the source code is closed. RIVeR may be suitable for real time monitoring thanks to the rectification of velocity vectors only. Overall, all the codes are found to be effective in performing Airborne Image Velocimetry in riverine environments. © 2020 International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLIII, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

Hyperspectral image classification has drawn significant attention in the recent years driven by the increasing abundance of sensor-generated hyper- and multi-spectral data, combined with the rapid advancements in the field of machine learning. A vast range of techniques, especially involving deep learning models, have been proposed attaining high levels of classification accuracy. However, many of these approaches significantly deteriorate in performance in the presence of noise in the hyperspectral data. In this paper, we propose a new model that effectively addresses the challenge of noise residing in hyperspectral images. The proposed model, which will be called DCT-CNN, combines the representational power of Convolutional Neural Networks with the noise elimination capabilities introduced by frequency-domain filtering enabled through the Discrete Cosine Transform. In particular, the proposed method entails the transformation of pixel macroblocks to the frequency domain and the discarding of information that corresponds to the higher frequencies in every patch, in which pixel information of abrupt changes and noise often resides. Experiment results in Indian Pines, Salinas and Pavia University datasets indicate that the proposed DCT-CNN constitutes a promising new model for accurate hyperspectral image classification offering robustness to different types of noise, such as Gaussian and salt and pepper noise. © 2020 International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLIII, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

The advantages of terrestrial laser scanning (TLS) for geodetic monitoring of man-made and natural objects are not yet fully exploited. Herein we address one of the open challenges by proposing feature-based methods for identification of corresponding points in point clouds of two or more epochs. We propose a learned compact feature descriptor tailored for point clouds of natural outdoor scenes obtained using TLS. We evaluate our method both on a benchmark data set and on a specially acquired outdoor dataset resembling a simplified monitoring scenario where we successfully estimate 3D displacement vectors of a rock that has been displaced between the scans. We show that the proposed descriptor has the capacity to generalize to unseen data and achieves state-of-the-art performance while being time efficient at the matching step due the low dimension., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, IV-2, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

We propose to use a discriminative classifier for outlier detection in large-scale point clouds of cities generated via multi-view stereo (MVS) from densely acquired images. What makes outlier removal hard are varying distributions of inliers and outliers across a scene. Heuristic outlier removal using a specific feature that encodes point distribution often delivers unsatisfying results. Although most outliers can be identified correctly (high recall), many inliers are erroneously removed (low precision), too. This aggravates object 3D reconstruction due to missing data. We thus propose to discriminatively learn class-specific distributions directly from the data to achieve high precision. We apply a standard Random Forest classifier that infers a binary label (inlier or outlier) for each 3D point in the raw, unfiltered point cloud and test two approaches for training. In the first, non-semantic approach, features are extracted without considering the semantic interpretation of the 3D points. The trained model approximates the average distribution of inliers and outliers across all semantic classes. Second, semantic interpretation is incorporated into the learning process, i.e. we train separate inlieroutlier classifiers per semantic class (building facades, roof, ground, vegetation, fields, and water). Performance of learned filtering is evaluated on several large SfM point clouds of cities. We find that results confirm our underlying assumption that discriminatively learning inlier-outlier distributions does improve precision over global heuristics by up to ≈ 12 percent points. Moreover, semantically informed filtering that models class-specific distributions further improves precision by up to ≈ 10 percent points, being able to remove very isolated building, roof, and water points while preserving inliers on building facades and vegetation., ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, IV-2, ISSN:2194-9042

An underwater imaging system with camera and lens behind a flat port does not behave as a standard pinhole camera with additional parameters. Indeed, whenever the entrance pupil of the lens is not in contact with the flat port, the standard photogrammetric model is not suited anymore and an extended mathematical model that considers the different media would be required. Therefore, when dealing with flat ports, the use of the classic photogrammetric formulation represents a simplification of the image formation phenomenon, clearly causing a degradation in accuracy. Furthermore, flat ports significantly change the characteristics of the enclosed imaging device and negatively affect the image quality, introducing heavy curvilinear distortions and optical aberrations. With the aim of mitigating the effect of systematic errors introduced by a combination of (i) image quality degradation, induced by the flat ports, and (ii) a non-rigorous modelling of refraction, this paper presents a stochastic model for image observations that penalises those that are more affected by aberrations and departure from the pinhole model. Experiments were carried out at sea and in pools showing that the use of the proposed stochastic model is beneficial for the final accuracy with improvements up to 50 %.

Ecological studies about marine benthic communities received a major leap from the application of a variety of non-destructive sampling and mapping techniques based on underwater image and video recording. The well-established scientific diving practice consists in the acquisition of single path or ‘round-trip’ over elongated transects, with the imaging device oriented in a nadir looking direction. As it may be expected, the application of automatic image processing procedures to data not specifically acquired for 3D modelling can be risky, especially if proper tools for assessing the quality of the produced results are not employed. This paper, born from an international cooperation, focuses on this topic, which is of great interest for ecological and monitoring benthic studies in Antarctica. Several video footages recorded from different scientific teams in different years are processed with an automatic photogrammetric procedure and salient statistical features are reported to critically analyse the derived results. As expected, the inclusion of oblique images from additional lateral strips may improve the expected accuracy in the object space, without altering too much the current video recording practices., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

This paper investigates the use of different greyscale conversion algorithms to decolourize colour images as input for two Structure-from-Motion (SfM) software packages. Although SfM software commonly works with a wide variety of frame imagery (old and new, colour and greyscale, airborne and terrestrial, large-and small scale), most programs internally convert the source imagery to single-band, greyscale images. This conversion is often assumed to have little, if any, impact on the final outcome. To verify this assumption, this article compares the output of an academic and a commercial SfM software package using seven different collections of architectural images. Besides the conventional 8-bit true-colour JPEG images with embedded sRGB colour profiles, for each of those datasets, 57 greyscale variants were computed with different colour-to-greyscale algorithms. The success rate of specific colour conversion approaches can therefore be compared with the commonly implemented colour-to-greyscale algorithms (luma Y’601, luma Y’709, or luminance CIE Y), both in terms of the applied feature extractor as well as of the specific image content (as exemplified by the two different feature descriptors and the various image collections, respectively). Although the differences can be small, the results clearly indicate that certain colour-to-greyscale conversion algorithms in an SfM-workflow constantly perform better than others. Overall, one of the best performing decolourization algorithms turns out to be a newly developed one.