Search

The use of Airborne Laser Scanner (ALS) point clouds has dominated 3D buildings reconstruction research, thus giving photogrammetric point clouds less attention. Point cloud density, occlusion and vegetation cover are some of the concerns that promote the necessity to understand and question the completeness and correctness of UAV photogrammetric point clouds for 3D buildings reconstruction. This research explores the potentials of modelling 3D buildings from nadir and oblique UAV image data vis a vis airborne laser data. Optimal parameter settings for dense matching and reconstruction are analysed for both UAV image-based and lidar point clouds. This research employs an automatic data driven model approach to 3D building reconstruction. A proper segmentation into planar roof faces is crucial, followed by façade detection to capture the real extent of the buildings’ roof overhang. An analysis of the quality of point density and point noise, in relation to setting parameter indicates that with a minimum of 50 points/m2, most of the planar surfaces are reconstructed comfortably. But for smaller features than dormers on the roof, a denser point cloud than 80 points/m2 is needed. 3D buildings from UAVs point cloud can be improved by enhancing roof boundary by use of edge information from images. It can also be improved by merging the imagery building outlines, point clouds roof boundary and the walls outline to extract the real extent of the building.

Precise and accurate three-dimensional geospatial data has become increasingly available thanks to advances in both Terrestrial Laser Scanning (TLS) and Structure-from-Motion Photogrammetry (SfM). These tools provide valuable information for mapping geomorphological features and detect surface changes in mountainous environments. The exploitation of 3D point-clouds has been proven tremendously useful in the field of geosciences. It remains, however, controversial whether cost efficient photogrammetry can provide as accurate and reliable geospatial information as the significantly more expensive laser scanning or not. In this study, a rockfall case site in the territory of Obergurgl, Austria, is investigated in order to provide answers to the above question in a complex environment. The analysis includes different terrestrial photogrammetry configurations aiming to comprehensively define the strengths and limitations of terrestrial photogrammetry over TLS. The latter constitutes an optimized methodology that provides guidelines for costly future assessments as part of the site investigation phase in geohazard management. There are no doubts that compared to traditional and conventional surveying methods TLS and Photogrammetry both offer products much faster and with a much higher data density. In the current study, we show that when photogrammetry is applied following a well-defined optimized strategy, it can be potentially an adequate alternative to more costly TLS datasets for mass movement assessment and monitoring purposes.

Deep learning methods based on Fully convolution networks (FCNs) have shown an impressive progress in building outline delineation from very high resolution (VHR) remote sensing (RS) imagery. Common issues still exist in extracting precise building shapes and outlines, often resulting in irregular edges and over smoothed corners. In this paper, we use PolyMapper, a recently introduced deep-learning framework that is able to predict object outlines in a vector representation directly. We have introduced two main modifications to this baseline method. First, we introduce EffcientNet as backbone feature encoder to our network, which uses compound coefficient to scale up all dimensions of depth/width/resolution uniformly, to improve the processing speed with fewer parameters. Second, we integrate a boundary refinement block (BRB) to strengthen the boundary feature learning and to further improve the accuracy of corner prediction. The results demonstrate that the end-to-end learnable model is capable of delineating polygons of building outlines that closely approximate the structure of reference labels. Experiments on the crowdAI building instance segmentation datasets show that our model outperforms PolyMapper in all COCO metrics, for instance showing a 0.13 higher mean Average Precision (AP) value and a 0.60 higher mean Average Recall value. Also qualitative results show that our method segments building instances of various shapes more accurately.

Semantic segmentation models are often affected by illumination changes, and fail to predict correct labels. Although there has been a lot of research on indoor semantic segmentation, it has not been studied in low-light environments. In this paper we propose a new framework, LISU, for Low-light Indoor Scene Understanding. We first decompose the low-light images into reflectance and illumination components, and then jointly learn reflectance restoration and semantic segmentation. To train and evaluate the proposed framework, we propose a new data set, namely LLRGBD, which consists of a large synthetic low-light indoor data set (LLRGBD-synthetic) and a small real data set (LLRGBD-real). The experimental results show that the illumination-invariant features effectively improve the performance of semantic segmentation. Compared with the baseline model, the mIoU of the proposed LISU framework has increased by 11.5%. In addition, pre-training on our synthetic data set increases the mIoU by 7.2%. Our data sets and models are available on our project website.

Deep learning-based semantic segmentation models for building delineation face the challenge of producing precise and regular building outlines. Recently, a building delineation method based on frame field learning was proposed by Girard et al. (2020) to extract regular building footprints as vector polygons directly from aerial RGB images. A fully convolution network (FCN) is trained to learn simultaneously the building mask, contours, and frame field followed by a polygonization method. With the direction information of the building contours stored in the frame field, the polygonization algorithm produces regular outlines accurately detecting edges and corners. This paper investigated the contribution of elevation data from the normalized digital surface model (nDSM) to extract accurate and regular building polygons. The 3D information provided by the nDSM overcomes the aerial images’ limitations and contributes to distinguishing the buildings from the background more accurately. Experiments conducted in Enschede, the Netherlands, demonstrate that the nDSM improves building outlines’ accuracy, resulting in better-aligned building polygons and prevents false positives. The investigated deep learning approach (fusing RGB + nDSM) results in a mean intersection over union (IOU) of 0.70 in the urban area. The baseline method (using RGB only) results in an IOU of 0.58 in the same area. A qualitative analysis of the results shows that the investigated model predicts more precise and regular polygons for large and complex structures.

The classification of Mobile Laser Scanner (MLS) data is challenging due to the combination of high variation in point density with a high variation of object appearances. The way how objects appear in the MLS data highly depends on the speed and orientation of the mobile mapping platform and the occlusion by other vehicles. There have been many approaches dealing with the geometric and contextual appearance of MLS points, voxels and segments to classify the MLS data. We present a completely different strategy by fusing the MLS data with a large scale topographic map. Underlying assumption is that the map delivers a clear hint on what to expect in the MLS data, at its approximate location. The approach presented here first fuses polygon objects, such as road, water, terrain and buildings, with ground and non-ground MLS points. Non-ground MLS points above roads and terrain are further classified by segmenting and matching the laser points to corresponding map point objects. The segmentation parameters depend on the class of the map points. We show that the fusion process is capable of classifying MLS data and detecting changes between the map and MLS data. The segmentation algorithm is not perfect, at some occasions not all the MLS points are correctly assigned to the corresponding map object. However, it is without doubt that the proposed map fusion delivers a very rich labelled point cloud automatically, which in future work can be used as training data in deep learning approaches.

With the development of LiDAR and photogrammetric techniques, more and more point clouds are available with high density and in large areas. Point cloud interpretation is an important step before many real applications like 3D city modelling. Many supervised machine learning techniques have been adapted to semantic point cloud segmentation, aiming to automatically label point clouds. Current deep learning methods have shown their potentials to produce high accuracy in semantic point cloud segmentation tasks. However, these supervised methods require a large amount of labelled data for proper model performance and good generalization. In practice, manual labelling of point clouds is very expensive and time-consuming. Active learning can iteratively select unlabelled samples for manual annotation based on current statistical models and then update the labelled data pool for next model training. In order to effectively label point clouds, we proposed a segment based active learning strategy to assess the informativeness of samples. Here, the proposed strategy uses 40% of the whole training dataset to achieve a mean IoU of 75.2% which is 99.1% of the accuracy in mIoU obtained from the model trained on the full dataset, while the baseline method using same amount of data only reaches 69.6% in mIoU corresponding to 90.9% of the accuracy in mIoU obtained from the model trained on the full dataset.

Degradation and damage detection provides essential information to maintenance workers in routine monitoring and to first responders in post-disaster scenarios. Despite advance in Earth Observation (EO), image analysis and deep learning techniques, the quality and quantity of training data for deep learning is still limited. As a result, no robust method has been found yet that can transfer and generalize well over a variety of geographic locations and typologies of damages. Since damages can be seen as anomalies, occurring sparingly over time and space, we propose to use an anomaly detecting Generative Adversarial Network (GAN) to detect damages. The main advantages of using GANs are that only healthy unannotated images are needed, and that a variety of damages, including the never before seen damage, can be detected. In this study we aimed to investigate 1) the ability of anomaly detecting GANs to detect degradation (potholes and cracks) in asphalt road infrastructures using Mobile Mapper imagery and building damage (collapsed buildings, rubble piles) using post-disaster aerial imagery, and 2) the sensitivity of this method against various types of pre-processing. Our results show that we can detect damages in urban scenes at satisfying levels but not on asphalt roads. Future work will investigate how to further classify the found damages and how to improve damage detection for asphalt roads.

Knowledge of tree species mapping and of dead wood in particular is fundamental to managing our forests. Although individual tree-based approaches using lidar can successfully distinguish between deciduous and coniferous trees, the classification of multiple tree species is still limited in accuracy. Moreover, the combined mapping of standing dead trees after pest infestation is becoming increasingly important. New deep learning methods outperform baseline machine learning approaches and promise a significant accuracy gain for tree mapping. In this study, we performed a classification of multiple tree species (pine, birch, alder) and standing dead trees with crowns using the 3D deep neural network (DNN) PointNet++ along with UAV-based lidar data and multispectral (MS) imagery. Aside from 3D geometry, we also integrated laser echo pulse width values and MS features into the classification process. In a preprocessing step, we generated the 3D segments of single trees using a 3D detection method. Our approach achieved an overall accuracy (OA) of 90.2% and was clearly superior to a baseline method using a random forest classifier and handcrafted features (OA = 85.3%). All in all, we demonstrate that the performance of the 3D DNN is highly promising for the classification of multiple tree species and standing dead trees in practice.

Modern machine learning, especially deep learning, which is used in a variety of applications, requires a lot of labelled data for model training. Having an insufficient amount of training examples leads to models which do not generalize well to new input instances. This is a particular significant problem for tasks involving aerial images: often training data is only available for a limited geographical area and a narrow time window, thus leading to models which perform poorly in different regions, at different times of day, or during different seasons. Domain adaptation can mitigate this issue by using labelled source domain training examples and unlabeled target domain images to train a model which performs well on both domains. Modern adversarial domain adaptation approaches use unpaired data. We propose using pairs of semantically similar images, i.e., whose segmentations are accurate predictions of each other, for improved model performance. In this paper we show that, as an upper limit based on ground truth, using semantically paired aerial images during training almost always increases model performance with an average improvement of 4.2% accuracy and .036 mean intersection-over-union (mIoU). Using a practical estimate of semantic similarity, we still achieve improvements in more than half of all cases, with average improvements of 2.5% accuracy and .017 mIoU in those cases.

Land use (LU) is an important information source commonly stored in geospatial databases. Most current work on automatic LU classification for updating topographic databases considers only one category level (e.g. residential or agricultural) consisting of a small number of classes. However, LU databases frequently contain very detailed information, using a hierarchical object catalogue where the number of categories differs depending on the hierarchy level. This paper presents a method for the classification of LU on the basis of aerial images that differentiates a fine-grained class structure, exploiting the hierarchical relationship between categories at different levels of the class catalogue. Starting from a convolutional neural network (CNN) for classifying the categories of all levels, we propose a strategy to simultaneously learn the semantic dependencies between different category levels explicitly. The input to the CNN consists of aerial images and derived data as well as land cover information derived from semantic segmentation. Its output is the class scores at three different semantic levels, based on which predictions that are consistent with the class hierarchy are made. We evaluate our method using two test sites and show how the classification accuracy depends on the semantic category level. While at the coarsest level, an overall accuracy in the order of 90% can be achieved, at the finest level, this accuracy is reduced to around 65%. Our experiments also show which classes are particularly hard to differentiate. © 2020 Copernicus GmbH. All rights reserved.

Background The CRASH-3 trial hypothesised that timely tranexamic acid (TXA) treatment might reduce deaths from intracranial bleeding after traumatic brain injury (TBI). To explore the mechanism of action of TXA in TBI, we examined the timing of its effect on death. Methods The CRASH-3 trial randomised 9202 patients within 3 h of injury with a GCS score ≤ 12 or intracranial bleeding on CT scan and no significant extracranial bleeding to receive TXA or placebo. We conducted an exploratory analysis of the effects of TXA on all-cause mortality within 24 h of injury and within 28 days, excluding patients with a GCS score of 3 or bilateral unreactive pupils, stratified by severity and country income. We pool data from the CRASH-2 and CRASH-3 trials in a one-step fixed effects individual patient data meta-analysis. Results There were 7637 patients for analysis after excluding patients with a GCS score of 3 or bilateral unreactive pupils. Of 1112 deaths, 23.3% were within 24 h of injury (early deaths). The risk of early death was reduced with TXA (112 (2.9%) TXA group vs 147 (3.9%) placebo group; risk ratio [RR] RR 0.74, 95% CI 0.58–0.94). There was no evidence of heterogeneity by severity (p = 0.64) or country income (p = 0.68). The risk of death beyond 24 h of injury was similar in the TXA and placebo groups (432 (11.5%) TXA group vs 421 (11.7%) placebo group; RR 0.98, 95% CI 0.69–1.12). The risk of death at 28 days was 14.0% in the TXA group versus 15.1% in the placebo group (544 vs 568 events; RR 0.93, 95% CI 0.83–1.03). When the CRASH-2 and CRASH-3 trial data were pooled, TXA reduced early death (RR 0.78, 95% CI 0.70–0.87) and death within 28 days (RR 0.88, 95% CI 0.82–0.94). Conclusions Tranexamic acid reduces early deaths in non-moribund TBI patients regardless of TBI severity or country income. The effect of tranexamic acid in patients with isolated TBI is similar to that in polytrauma. Treatment is safe and even severely injured patients appear to benefit when treated soon after injury. Trial registration ISRCTN15088122, registered on 19 July 2011; NCT01402882, registered on 26 July 2011.

Depth is an essential component for various scene understanding tasks and for reconstructing the 3D geometry of the scene. Estimating depth from stereo images requires multiple views of the same scene to be captured which is often not possible when exploring new environments with a UAV. To overcome this monocular depth estimation has been a topic of interest with the recent advancements in computer vision and deep learning techniques. This research has been widely focused on indoor scenes or outdoor scenes captured at ground level. Single image depth estimation from aerial images has been limited due to additional complexities arising from increased camera distance, wider area coverage with lots of occlusions. A new aerial image dataset is prepared specifically for this purpose combining Unmanned Aerial Vehicles (UAV) images covering different regions, features and point of views. The single image depth estimation is based on image reconstruction techniques which uses stereo images for learning to estimate depth from single images. Among the various available models for ground-level single image depth estimation, two models, 1) a Convolutional Neural Network (CNN) and 2) a Generative Adversarial model (GAN) are used to learn depth from aerial images from UAVs. These models generate pixel-wise disparity images which could be converted into depth information. The generated disparity maps from these models are evaluated for its internal quality using various error metrics. The results show higher disparity ranges with smoother images generated by CNN model and sharper images with lesser disparity range generated by GAN model. The produced disparity images are converted to depth information and compared with point clouds obtained using Pix4D. It is found that the CNN model performs better than GAN and produces depth similar to that of Pix4D. This comparison helps in streamlining the efforts to produce depth from a single aerial image.

In recent years, the determination of global image orientation, i.e. global SfM, has gained a lot of attentions from researchers, mainly due to its time efficiency. Most of the global methods take relative rotations and translations as input for a two-step strategy comprised of global rotation averaging and global translation averaging. This paper by contrast presents a hybrid approach that aims to solve global rotations and translations simultaneously, but hierarchically. We first extract an optimal minimum cover connected image triplet set (OMCTS) which includes all available images with a minimum number of triplets, all of them with the three related relative orientations being compatible to each other. For non-collinear triplets in the OMCTS, we introduce some basic characterizations of the corresponding essential matrices and solve for the image pose parameters by averaging the constrained essential matrices. For the collinear triplets, on the other hand, the image pose parameters are estimated by relative orientation using the depth of object points from individual local spatial intersection. Finally, all image orientations are estimated in a common coordinate frame by traversing every solved triplet using a similarity transformation. We show results of our method on different benchmarks and demonstrate the performance and capability of the proposed approach by comparing with other global SfM methods.

Many countries were the target of air strikes during World War II. Numerous unexploded bombs still exist in the ground. These duds can be tracked down with the help of bomb craters, indicating areas where unexploded bombs may be located. Such areas are documented in so-called impact maps based on detected bomb craters. In this paper, a stochastic approach based on marked point processes (MPPs) for the automatic detection of bomb craters in aerial images taken during World War II is presented. As most areas are covered by multiple images, the influence of redundant image information on the object detection result is investigated: we compare the results generated based on single images with those obtained by our new approach that combines the individual detection results of multiple images covering the same location. The object model for the bomb craters is represented by circles. Our MPP approach determines the most likely configuration of objects within the scene. The goal is reached by minimizing an energy function that describes the conformity with a predefined model by Reversible Jump Markov Chain Monte Carlo sampling in combination with simulated annealing. Afterwards, a probability map is generated from the automatic detections via kernel density estimation. By setting a threshold, areas around the detections are classified as contaminated or uncontaminated sites, respectively, which results in an impact map. Our results show a significant improvement with respect to its quality when redundant image information is used.

Fully convolutional neural networks (FCN) are successfully used for the automated pixel-wise classification of aerial images and possibly additional data. However, they require many labelled training samples to perform well. One approach addressing this issue is semi-supervised domain adaptation (SSDA). Here, labelled training samples from a source domain and unlabelled samples from a target domain are used jointly to obtain a target domain classifier, without requiring any labelled samples from the target domain. In this paper, a two-step approach for SSDA is proposed. The first step corresponds to a supervised training on the source domain, making use of strong data augmentation to increase the initial performance on the target domain. Secondly, the model is adapted by entropy minimization using a novel weighting strategy. The approach is evaluated on the basis of five domains, corresponding to five cities. Several training variants and adaptation scenarios are tested, indicating that proper data augmentation can already improve the initial target domain performance significantly resulting in an average overall accuracy of 77.5%. The weighted entropy minimization improves the overall accuracy on the target domains in 19 out of 20 scenarios on average by 1.8%. In all experiments a novel FCN architecture is used that yields results comparable to those of the best-performing models on the ISPRS labelling challenge while having an order of magnitude fewer parameters than commonly used FCNs.

This paper proposes several methods for training a Convolutional Neural Network (CNN) for learning the similarity between images of silk fabrics based on multiple semantic properties of the fabrics. In the context of the EU H2020 project SILKNOW (http://silknow.eu/), two variants of training were developed, one based on a Siamese CNN and one based on a triplet architecture. We propose different definitions of similarity and different loss functions for both training strategies, some of them also allowing the use of incomplete information about the training data. We assess the quality of the trained model by using the learned image features in a k-NN classification. We achieve overall accuracies of 93–95% and average F1-scores of 87–92%.

Matching images containing large viewpoint and viewing direction changes, resulting in large perspective differences, still is a very challenging problem. Affine shape estimation, orientation assignment and feature description algorithms based on detected hand crafted features have shown to be error prone. In this paper, affine shape estimation, orientation assignment and description of local features is achieved through deep learning. Those three modules are trained based on loss functions optimizing the matching performance of input patch pairs. The trained descriptors are first evaluated on the Brown dataset (Brown et al., 2011), a standard descriptor performance benchmark. The whole pipeline is then tested on images of small blocks acquired with an aerial penta camera, to compute image orientation. The results show that learned features perform significantly better than alternatives based on hand crafted features. © 2020 Copernicus GmbH. All rights reserved.

State-of-the-art object detection approaches such as Fast/Faster R-CNN, SSD, or YOLO have difficulties detecting dense, small targets with arbitrary orientation in large aerial images. The main reason is that using interpolation to align RoI features can result in a lack of accuracy or even loss of location information. We present the Local-aware Region Convolutional Neural Network (LR-CNN), a novel two-stage approach for vehicle detection in aerial imagery. We enhance translation invariance to detect dense vehicles and address the boundary quantization issue amongst dense vehicles by aggregating the high-precision RoIs' features. Moreover, we resample high-level semantic pooled features, making them regain location information from the features of a shallower convolutional block. This strengthens the local feature invariance for the resampled features and enables detecting vehicles in an arbitrary orientation. The local feature invariance enhances the learning ability of the focal loss function, and the focal loss further helps to focus on the hard examples. Taken together, our method better addresses the challenges of aerial imagery. We evaluate our approach on several challenging datasets (VEDAI, DOTA), demonstrating a significant improvement over state-of-the-art methods. We demonstrate the good generalization ability of our approach on the DLR 3K dataset., Comment: 8 pages

In this paper we present a novel approach for image orientation by combining relative rotations and tie points. First, we choose an initial image pair with enough correspondences and large triangulation angle, and we then iteratively add clusters of new images. The rotation of these newly added images is estimated from relative rotations by single rotation averaging. In the next step, a linear equation system is set up for each new image to solve the translation parameters with triangulated tie points which can be viewed in that new image, followed by a resection for refinement. Finally, we optimize the cluster of reconstructed images by local bundle adjustment. We show results of our approach on different benchmark datasets. Furthermore, we orient several larger datasets incl. unordered image datasets to demonstrate the robustness and performance of our approach.

In this paper we deal with the problem of measuring the similarity between training and tests datasets in the context of transfer learning (TL) for image classification. TL tries to transfer knowledge from a source domain, where labelled training samples are abundant but the data may follow a different distribution, to a target domain, where labelled training samples are scarce or even unavailable, assuming that the domains are related. Thus, the requirements w.r.t. the availability of labelled training samples in the target domain are reduced. In particular, if no labelled target data are available, it is inherently difficult to find a robust measure of relatedness between the source and target domains. This is of crucial importance for the performance of TL, because the knowledge transfer between unrelated data may lead to negative transfer, i.e. to a decrease of classification performance after transfer. We address the problem of measuring the relatedness between source and target datasets and investigate three different strategies to predict and, consequently, to avoid negative transfer in this paper. The first strategy is based on circular validation. The second strategy relies on the Maximum Mean Discrepancy (MMD) similarity metric, whereas the third one is an extension of MMD which incorporates the knowledge about the class labels in the source domain. Our method is evaluated using two different benchmark datasets. The experiments highlight the strengths and weaknesses of the investigated methods. We also show that it is possible to reduce the amount of negative transfer using these strategies for a TL method and to generate a consistent performance improvement over the whole dataset.