Search

The use of Airborne Laser Scanner (ALS) point clouds has dominated 3D buildings reconstruction research, thus giving photogrammetric point clouds less attention. Point cloud density, occlusion and vegetation cover are some of the concerns that promote the necessity to understand and question the completeness and correctness of UAV photogrammetric point clouds for 3D buildings reconstruction. This research explores the potentials of modelling 3D buildings from nadir and oblique UAV image data vis a vis airborne laser data. Optimal parameter settings for dense matching and reconstruction are analysed for both UAV image-based and lidar point clouds. This research employs an automatic data driven model approach to 3D building reconstruction. A proper segmentation into planar roof faces is crucial, followed by façade detection to capture the real extent of the buildings’ roof overhang. An analysis of the quality of point density and point noise, in relation to setting parameter indicates that with a minimum of 50 points/m2, most of the planar surfaces are reconstructed comfortably. But for smaller features than dormers on the roof, a denser point cloud than 80 points/m2 is needed. 3D buildings from UAVs point cloud can be improved by enhancing roof boundary by use of edge information from images. It can also be improved by merging the imagery building outlines, point clouds roof boundary and the walls outline to extract the real extent of the building.

Precise and accurate three-dimensional geospatial data has become increasingly available thanks to advances in both Terrestrial Laser Scanning (TLS) and Structure-from-Motion Photogrammetry (SfM). These tools provide valuable information for mapping geomorphological features and detect surface changes in mountainous environments. The exploitation of 3D point-clouds has been proven tremendously useful in the field of geosciences. It remains, however, controversial whether cost efficient photogrammetry can provide as accurate and reliable geospatial information as the significantly more expensive laser scanning or not. In this study, a rockfall case site in the territory of Obergurgl, Austria, is investigated in order to provide answers to the above question in a complex environment. The analysis includes different terrestrial photogrammetry configurations aiming to comprehensively define the strengths and limitations of terrestrial photogrammetry over TLS. The latter constitutes an optimized methodology that provides guidelines for costly future assessments as part of the site investigation phase in geohazard management. There are no doubts that compared to traditional and conventional surveying methods TLS and Photogrammetry both offer products much faster and with a much higher data density. In the current study, we show that when photogrammetry is applied following a well-defined optimized strategy, it can be potentially an adequate alternative to more costly TLS datasets for mass movement assessment and monitoring purposes.

Deep learning methods based on Fully convolution networks (FCNs) have shown an impressive progress in building outline delineation from very high resolution (VHR) remote sensing (RS) imagery. Common issues still exist in extracting precise building shapes and outlines, often resulting in irregular edges and over smoothed corners. In this paper, we use PolyMapper, a recently introduced deep-learning framework that is able to predict object outlines in a vector representation directly. We have introduced two main modifications to this baseline method. First, we introduce EffcientNet as backbone feature encoder to our network, which uses compound coefficient to scale up all dimensions of depth/width/resolution uniformly, to improve the processing speed with fewer parameters. Second, we integrate a boundary refinement block (BRB) to strengthen the boundary feature learning and to further improve the accuracy of corner prediction. The results demonstrate that the end-to-end learnable model is capable of delineating polygons of building outlines that closely approximate the structure of reference labels. Experiments on the crowdAI building instance segmentation datasets show that our model outperforms PolyMapper in all COCO metrics, for instance showing a 0.13 higher mean Average Precision (AP) value and a 0.60 higher mean Average Recall value. Also qualitative results show that our method segments building instances of various shapes more accurately.

Semantic segmentation models are often affected by illumination changes, and fail to predict correct labels. Although there has been a lot of research on indoor semantic segmentation, it has not been studied in low-light environments. In this paper we propose a new framework, LISU, for Low-light Indoor Scene Understanding. We first decompose the low-light images into reflectance and illumination components, and then jointly learn reflectance restoration and semantic segmentation. To train and evaluate the proposed framework, we propose a new data set, namely LLRGBD, which consists of a large synthetic low-light indoor data set (LLRGBD-synthetic) and a small real data set (LLRGBD-real). The experimental results show that the illumination-invariant features effectively improve the performance of semantic segmentation. Compared with the baseline model, the mIoU of the proposed LISU framework has increased by 11.5%. In addition, pre-training on our synthetic data set increases the mIoU by 7.2%. Our data sets and models are available on our project website.

Deep learning-based semantic segmentation models for building delineation face the challenge of producing precise and regular building outlines. Recently, a building delineation method based on frame field learning was proposed by Girard et al. (2020) to extract regular building footprints as vector polygons directly from aerial RGB images. A fully convolution network (FCN) is trained to learn simultaneously the building mask, contours, and frame field followed by a polygonization method. With the direction information of the building contours stored in the frame field, the polygonization algorithm produces regular outlines accurately detecting edges and corners. This paper investigated the contribution of elevation data from the normalized digital surface model (nDSM) to extract accurate and regular building polygons. The 3D information provided by the nDSM overcomes the aerial images’ limitations and contributes to distinguishing the buildings from the background more accurately. Experiments conducted in Enschede, the Netherlands, demonstrate that the nDSM improves building outlines’ accuracy, resulting in better-aligned building polygons and prevents false positives. The investigated deep learning approach (fusing RGB + nDSM) results in a mean intersection over union (IOU) of 0.70 in the urban area. The baseline method (using RGB only) results in an IOU of 0.58 in the same area. A qualitative analysis of the results shows that the investigated model predicts more precise and regular polygons for large and complex structures.

The classification of Mobile Laser Scanner (MLS) data is challenging due to the combination of high variation in point density with a high variation of object appearances. The way how objects appear in the MLS data highly depends on the speed and orientation of the mobile mapping platform and the occlusion by other vehicles. There have been many approaches dealing with the geometric and contextual appearance of MLS points, voxels and segments to classify the MLS data. We present a completely different strategy by fusing the MLS data with a large scale topographic map. Underlying assumption is that the map delivers a clear hint on what to expect in the MLS data, at its approximate location. The approach presented here first fuses polygon objects, such as road, water, terrain and buildings, with ground and non-ground MLS points. Non-ground MLS points above roads and terrain are further classified by segmenting and matching the laser points to corresponding map point objects. The segmentation parameters depend on the class of the map points. We show that the fusion process is capable of classifying MLS data and detecting changes between the map and MLS data. The segmentation algorithm is not perfect, at some occasions not all the MLS points are correctly assigned to the corresponding map object. However, it is without doubt that the proposed map fusion delivers a very rich labelled point cloud automatically, which in future work can be used as training data in deep learning approaches.

With the development of LiDAR and photogrammetric techniques, more and more point clouds are available with high density and in large areas. Point cloud interpretation is an important step before many real applications like 3D city modelling. Many supervised machine learning techniques have been adapted to semantic point cloud segmentation, aiming to automatically label point clouds. Current deep learning methods have shown their potentials to produce high accuracy in semantic point cloud segmentation tasks. However, these supervised methods require a large amount of labelled data for proper model performance and good generalization. In practice, manual labelling of point clouds is very expensive and time-consuming. Active learning can iteratively select unlabelled samples for manual annotation based on current statistical models and then update the labelled data pool for next model training. In order to effectively label point clouds, we proposed a segment based active learning strategy to assess the informativeness of samples. Here, the proposed strategy uses 40% of the whole training dataset to achieve a mean IoU of 75.2% which is 99.1% of the accuracy in mIoU obtained from the model trained on the full dataset, while the baseline method using same amount of data only reaches 69.6% in mIoU corresponding to 90.9% of the accuracy in mIoU obtained from the model trained on the full dataset.

Degradation and damage detection provides essential information to maintenance workers in routine monitoring and to first responders in post-disaster scenarios. Despite advance in Earth Observation (EO), image analysis and deep learning techniques, the quality and quantity of training data for deep learning is still limited. As a result, no robust method has been found yet that can transfer and generalize well over a variety of geographic locations and typologies of damages. Since damages can be seen as anomalies, occurring sparingly over time and space, we propose to use an anomaly detecting Generative Adversarial Network (GAN) to detect damages. The main advantages of using GANs are that only healthy unannotated images are needed, and that a variety of damages, including the never before seen damage, can be detected. In this study we aimed to investigate 1) the ability of anomaly detecting GANs to detect degradation (potholes and cracks) in asphalt road infrastructures using Mobile Mapper imagery and building damage (collapsed buildings, rubble piles) using post-disaster aerial imagery, and 2) the sensitivity of this method against various types of pre-processing. Our results show that we can detect damages in urban scenes at satisfying levels but not on asphalt roads. Future work will investigate how to further classify the found damages and how to improve damage detection for asphalt roads.

Knowledge of tree species mapping and of dead wood in particular is fundamental to managing our forests. Although individual tree-based approaches using lidar can successfully distinguish between deciduous and coniferous trees, the classification of multiple tree species is still limited in accuracy. Moreover, the combined mapping of standing dead trees after pest infestation is becoming increasingly important. New deep learning methods outperform baseline machine learning approaches and promise a significant accuracy gain for tree mapping. In this study, we performed a classification of multiple tree species (pine, birch, alder) and standing dead trees with crowns using the 3D deep neural network (DNN) PointNet++ along with UAV-based lidar data and multispectral (MS) imagery. Aside from 3D geometry, we also integrated laser echo pulse width values and MS features into the classification process. In a preprocessing step, we generated the 3D segments of single trees using a 3D detection method. Our approach achieved an overall accuracy (OA) of 90.2% and was clearly superior to a baseline method using a random forest classifier and handcrafted features (OA = 85.3%). All in all, we demonstrate that the performance of the 3D DNN is highly promising for the classification of multiple tree species and standing dead trees in practice.

Modern machine learning, especially deep learning, which is used in a variety of applications, requires a lot of labelled data for model training. Having an insufficient amount of training examples leads to models which do not generalize well to new input instances. This is a particular significant problem for tasks involving aerial images: often training data is only available for a limited geographical area and a narrow time window, thus leading to models which perform poorly in different regions, at different times of day, or during different seasons. Domain adaptation can mitigate this issue by using labelled source domain training examples and unlabeled target domain images to train a model which performs well on both domains. Modern adversarial domain adaptation approaches use unpaired data. We propose using pairs of semantically similar images, i.e., whose segmentations are accurate predictions of each other, for improved model performance. In this paper we show that, as an upper limit based on ground truth, using semantically paired aerial images during training almost always increases model performance with an average improvement of 4.2% accuracy and .036 mean intersection-over-union (mIoU). Using a practical estimate of semantic similarity, we still achieve improvements in more than half of all cases, with average improvements of 2.5% accuracy and .017 mIoU in those cases.

Land use (LU) is an important information source commonly stored in geospatial databases. Most current work on automatic LU classification for updating topographic databases considers only one category level (e.g. residential or agricultural) consisting of a small number of classes. However, LU databases frequently contain very detailed information, using a hierarchical object catalogue where the number of categories differs depending on the hierarchy level. This paper presents a method for the classification of LU on the basis of aerial images that differentiates a fine-grained class structure, exploiting the hierarchical relationship between categories at different levels of the class catalogue. Starting from a convolutional neural network (CNN) for classifying the categories of all levels, we propose a strategy to simultaneously learn the semantic dependencies between different category levels explicitly. The input to the CNN consists of aerial images and derived data as well as land cover information derived from semantic segmentation. Its output is the class scores at three different semantic levels, based on which predictions that are consistent with the class hierarchy are made. We evaluate our method using two test sites and show how the classification accuracy depends on the semantic category level. While at the coarsest level, an overall accuracy in the order of 90% can be achieved, at the finest level, this accuracy is reduced to around 65%. Our experiments also show which classes are particularly hard to differentiate. © 2020 Copernicus GmbH. All rights reserved.

Background The CRASH-3 trial hypothesised that timely tranexamic acid (TXA) treatment might reduce deaths from intracranial bleeding after traumatic brain injury (TBI). To explore the mechanism of action of TXA in TBI, we examined the timing of its effect on death. Methods The CRASH-3 trial randomised 9202 patients within 3 h of injury with a GCS score ≤ 12 or intracranial bleeding on CT scan and no significant extracranial bleeding to receive TXA or placebo. We conducted an exploratory analysis of the effects of TXA on all-cause mortality within 24 h of injury and within 28 days, excluding patients with a GCS score of 3 or bilateral unreactive pupils, stratified by severity and country income. We pool data from the CRASH-2 and CRASH-3 trials in a one-step fixed effects individual patient data meta-analysis. Results There were 7637 patients for analysis after excluding patients with a GCS score of 3 or bilateral unreactive pupils. Of 1112 deaths, 23.3% were within 24 h of injury (early deaths). The risk of early death was reduced with TXA (112 (2.9%) TXA group vs 147 (3.9%) placebo group; risk ratio [RR] RR 0.74, 95% CI 0.58–0.94). There was no evidence of heterogeneity by severity (p = 0.64) or country income (p = 0.68). The risk of death beyond 24 h of injury was similar in the TXA and placebo groups (432 (11.5%) TXA group vs 421 (11.7%) placebo group; RR 0.98, 95% CI 0.69–1.12). The risk of death at 28 days was 14.0% in the TXA group versus 15.1% in the placebo group (544 vs 568 events; RR 0.93, 95% CI 0.83–1.03). When the CRASH-2 and CRASH-3 trial data were pooled, TXA reduced early death (RR 0.78, 95% CI 0.70–0.87) and death within 28 days (RR 0.88, 95% CI 0.82–0.94). Conclusions Tranexamic acid reduces early deaths in non-moribund TBI patients regardless of TBI severity or country income. The effect of tranexamic acid in patients with isolated TBI is similar to that in polytrauma. Treatment is safe and even severely injured patients appear to benefit when treated soon after injury. Trial registration ISRCTN15088122, registered on 19 July 2011; NCT01402882, registered on 26 July 2011.

Depth is an essential component for various scene understanding tasks and for reconstructing the 3D geometry of the scene. Estimating depth from stereo images requires multiple views of the same scene to be captured which is often not possible when exploring new environments with a UAV. To overcome this monocular depth estimation has been a topic of interest with the recent advancements in computer vision and deep learning techniques. This research has been widely focused on indoor scenes or outdoor scenes captured at ground level. Single image depth estimation from aerial images has been limited due to additional complexities arising from increased camera distance, wider area coverage with lots of occlusions. A new aerial image dataset is prepared specifically for this purpose combining Unmanned Aerial Vehicles (UAV) images covering different regions, features and point of views. The single image depth estimation is based on image reconstruction techniques which uses stereo images for learning to estimate depth from single images. Among the various available models for ground-level single image depth estimation, two models, 1) a Convolutional Neural Network (CNN) and 2) a Generative Adversarial model (GAN) are used to learn depth from aerial images from UAVs. These models generate pixel-wise disparity images which could be converted into depth information. The generated disparity maps from these models are evaluated for its internal quality using various error metrics. The results show higher disparity ranges with smoother images generated by CNN model and sharper images with lesser disparity range generated by GAN model. The produced disparity images are converted to depth information and compared with point clouds obtained using Pix4D. It is found that the CNN model performs better than GAN and produces depth similar to that of Pix4D. This comparison helps in streamlining the efforts to produce depth from a single aerial image.

Depth is an essential component for various scene understanding tasks and for reconstructing the 3D geometry of the scene. Estimating depth from stereo images requires multiple views of the same scene to be captured which is often not possible when exploring new environments with a UAV. To overcome this monocular depth estimation has been a topic of interest with the recent advancements in computer vision and deep learning techniques. This research has been widely focused on indoor scenes or outdoor scenes captured at ground level. Single image depth estimation from aerial images has been limited due to additional complexities arising from increased camera distance, wider area coverage with lots of occlusions. A new aerial image dataset is prepared specifically for this purpose combining Unmanned Aerial Vehicles (UAV) images covering different regions, features and point of views. The single image depth estimation is based on image reconstruction techniques which uses stereo images for learning to estimate depth from single images. Among the various available models for ground-level single image depth estimation, two models, 1) a Convolutional Neural Network (CNN) and 2) a Generative Adversarial model (GAN) are used to learn depth from aerial images from UAVs. These models generate pixel-wise disparity images which could be converted into depth information. The generated disparity maps from these models are evaluated for its internal quality using various error metrics. The results show higher disparity ranges with smoother images generated by CNN model and sharper images with lesser disparity range generated by GAN model. The produced disparity images are converted to depth information and compared with point clouds obtained using Pix4D. It is found that the CNN model performs better than GAN and produces depth similar to that of Pix4D. This comparison helps in streamlining the efforts to produce depth from a single aerial image.

In recent years, the determination of global image orientation, i.e. global SfM, has gained a lot of attentions from researchers, mainly due to its time efficiency. Most of the global methods take relative rotations and translations as input for a two-step strategy comprised of global rotation averaging and global translation averaging. This paper by contrast presents a hybrid approach that aims to solve global rotations and translations simultaneously, but hierarchically. We first extract an optimal minimum cover connected image triplet set (OMCTS) which includes all available images with a minimum number of triplets, all of them with the three related relative orientations being compatible to each other. For non-collinear triplets in the OMCTS, we introduce some basic characterizations of the corresponding essential matrices and solve for the image pose parameters by averaging the constrained essential matrices. For the collinear triplets, on the other hand, the image pose parameters are estimated by relative orientation using the depth of object points from individual local spatial intersection. Finally, all image orientations are estimated in a common coordinate frame by traversing every solved triplet using a similarity transformation. We show results of our method on different benchmarks and demonstrate the performance and capability of the proposed approach by comparing with other global SfM methods.

Many countries were the target of air strikes during World War II. Numerous unexploded bombs still exist in the ground. These duds can be tracked down with the help of bomb craters, indicating areas where unexploded bombs may be located. Such areas are documented in so-called impact maps based on detected bomb craters. In this paper, a stochastic approach based on marked point processes (MPPs) for the automatic detection of bomb craters in aerial images taken during World War II is presented. As most areas are covered by multiple images, the influence of redundant image information on the object detection result is investigated: we compare the results generated based on single images with those obtained by our new approach that combines the individual detection results of multiple images covering the same location. The object model for the bomb craters is represented by circles. Our MPP approach determines the most likely configuration of objects within the scene. The goal is reached by minimizing an energy function that describes the conformity with a predefined model by Reversible Jump Markov Chain Monte Carlo sampling in combination with simulated annealing. Afterwards, a probability map is generated from the automatic detections via kernel density estimation. By setting a threshold, areas around the detections are classified as contaminated or uncontaminated sites, respectively, which results in an impact map. Our results show a significant improvement with respect to its quality when redundant image information is used.

Fully convolutional neural networks (FCN) are successfully used for the automated pixel-wise classification of aerial images and possibly additional data. However, they require many labelled training samples to perform well. One approach addressing this issue is semi-supervised domain adaptation (SSDA). Here, labelled training samples from a source domain and unlabelled samples from a target domain are used jointly to obtain a target domain classifier, without requiring any labelled samples from the target domain. In this paper, a two-step approach for SSDA is proposed. The first step corresponds to a supervised training on the source domain, making use of strong data augmentation to increase the initial performance on the target domain. Secondly, the model is adapted by entropy minimization using a novel weighting strategy. The approach is evaluated on the basis of five domains, corresponding to five cities. Several training variants and adaptation scenarios are tested, indicating that proper data augmentation can already improve the initial target domain performance significantly resulting in an average overall accuracy of 77.5%. The weighted entropy minimization improves the overall accuracy on the target domains in 19 out of 20 scenarios on average by 1.8%. In all experiments a novel FCN architecture is used that yields results comparable to those of the best-performing models on the ISPRS labelling challenge while having an order of magnitude fewer parameters than commonly used FCNs.

This paper proposes several methods for training a Convolutional Neural Network (CNN) for learning the similarity between images of silk fabrics based on multiple semantic properties of the fabrics. In the context of the EU H2020 project SILKNOW (http://silknow.eu/), two variants of training were developed, one based on a Siamese CNN and one based on a triplet architecture. We propose different definitions of similarity and different loss functions for both training strategies, some of them also allowing the use of incomplete information about the training data. We assess the quality of the trained model by using the learned image features in a k-NN classification. We achieve overall accuracies of 93–95% and average F1-scores of 87–92%.

Matching images containing large viewpoint and viewing direction changes, resulting in large perspective differences, still is a very challenging problem. Affine shape estimation, orientation assignment and feature description algorithms based on detected hand crafted features have shown to be error prone. In this paper, affine shape estimation, orientation assignment and description of local features is achieved through deep learning. Those three modules are trained based on loss functions optimizing the matching performance of input patch pairs. The trained descriptors are first evaluated on the Brown dataset (Brown et al., 2011), a standard descriptor performance benchmark. The whole pipeline is then tested on images of small blocks acquired with an aerial penta camera, to compute image orientation. The results show that learned features perform significantly better than alternatives based on hand crafted features. © 2020 Copernicus GmbH. All rights reserved.

State-of-the-art object detection approaches such as Fast/Faster R-CNN, SSD, or YOLO have difficulties detecting dense, small targets with arbitrary orientation in large aerial images. The main reason is that using interpolation to align RoI features can result in a lack of accuracy or even loss of location information. We present the Local-aware Region Convolutional Neural Network (LR-CNN), a novel two-stage approach for vehicle detection in aerial imagery. We enhance translation invariance to detect dense vehicles and address the boundary quantization issue amongst dense vehicles by aggregating the high-precision RoIs' features. Moreover, we resample high-level semantic pooled features, making them regain location information from the features of a shallower convolutional block. This strengthens the local feature invariance for the resampled features and enables detecting vehicles in an arbitrary orientation. The local feature invariance enhances the learning ability of the focal loss function, and the focal loss further helps to focus on the hard examples. Taken together, our method better addresses the challenges of aerial imagery. We evaluate our approach on several challenging datasets (VEDAI, DOTA), demonstrating a significant improvement over state-of-the-art methods. We demonstrate the good generalization ability of our approach on the DLR 3K dataset., Comment: 8 pages

In this paper we present a novel approach for image orientation by combining relative rotations and tie points. First, we choose an initial image pair with enough correspondences and large triangulation angle, and we then iteratively add clusters of new images. The rotation of these newly added images is estimated from relative rotations by single rotation averaging. In the next step, a linear equation system is set up for each new image to solve the translation parameters with triangulated tie points which can be viewed in that new image, followed by a resection for refinement. Finally, we optimize the cluster of reconstructed images by local bundle adjustment. We show results of our approach on different benchmark datasets. Furthermore, we orient several larger datasets incl. unordered image datasets to demonstrate the robustness and performance of our approach.

In this paper we deal with the problem of measuring the similarity between training and tests datasets in the context of transfer learning (TL) for image classification. TL tries to transfer knowledge from a source domain, where labelled training samples are abundant but the data may follow a different distribution, to a target domain, where labelled training samples are scarce or even unavailable, assuming that the domains are related. Thus, the requirements w.r.t. the availability of labelled training samples in the target domain are reduced. In particular, if no labelled target data are available, it is inherently difficult to find a robust measure of relatedness between the source and target domains. This is of crucial importance for the performance of TL, because the knowledge transfer between unrelated data may lead to negative transfer, i.e. to a decrease of classification performance after transfer. We address the problem of measuring the relatedness between source and target datasets and investigate three different strategies to predict and, consequently, to avoid negative transfer in this paper. The first strategy is based on circular validation. The second strategy relies on the Maximum Mean Discrepancy (MMD) similarity metric, whereas the third one is an extension of MMD which incorporates the knowledge about the class labels in the source domain. Our method is evaluated using two different benchmark datasets. The experiments highlight the strengths and weaknesses of the investigated methods. We also show that it is possible to reduce the amount of negative transfer using these strategies for a TL method and to generate a consistent performance improvement over the whole dataset.

Image-based modeling techniques are an important tool for producing 3D models in a practical and cost effective manner. Accurate image-based models can be created as long as one can retrieve precise image calibration and orientation information which is nowadays performed automatically in computer vision and photogrammetry. The first step for orientation is to have sufficient correspondences across the captured images. Keypoint descriptors like SIFT or SURF are a successful approach for finding these correspondences. The extraction of precise image correspondences is crucial for the subsequent image orientation and image matching steps. Indeed there are still many challenges especially with wide-baseline image configuration. After the extraction of a sufficient and reliable set of image correspondences, a bundle adjustment is used to retrieve the image orientation parameters. In this paper, a brief description of our previous work on automatic camera network design is initially reported. This semi-automatic procedure results in wide-baseline high resolution images covering an object of interest, and including approximations of image orientations, a rough 3D object geometry and a matching matrix indicating for each image its matching mates. The main part of this paper will describe the subsequent image matching where the pre-knowledge on the image orientations and the pre-created rough 3D model of the study object is exploited. Ultimately the matching information retrieved during that step will be used for a precise bundle block adjustment. Since we defined the initial image orientation in the design of the network, we can compute the matching matrix prior to image matching of high resolution images. For each image involved in several pairs that is defined in the matching matrix, we detect the corners or keypoints and then transform them into the matching images by using the designed orientation and initial 3D model. Moreover, a window is defined for each corner and its initial correspondence in the matching images. A SIFT or SURF matching is implemented between every matching window to find the homologous points. This is followed by Least Square Matching LSM to refine the correspondences for a sub-pixel localization and to avoid inaccurate matches. Image matching is followed by a bundle adjustment to orient the images automatically to finally have a sparse 3D model. We used the commercial software Photomodeler Scanner 2010 for implementing the bundle adjustment since it reports a number of accuracy indices which are necessary for the evaluation purposes. The experimental test of comparing the automated image matching of four pre-designed streopairs shows that our approach can provide a high accuracy and effective orientation when compared to the results of commercial and open source software which does not exploit the pre-knowledge about the scene.

We have been developing a new space dosimeter named "Position- Sensitive Tissue-Equivalent Proportional Chamber (PS-TEPC)" with tissue equivalency and position sensitivity. The detector is a gas time projection chamber filled with methane-based tissue equivalent gas. The Bread Board Model (BBM) for the detection part was manufactured and irradiated. Data acquisition for energy deposition and three dimensional tracks has been performed.

International audience; We present a summary of current efforts to find a Kuiper Belt Object that is within reach of the New Horizons spacecraft after its encounter with Pluto.

Earth- and space-based observations provide synergistic information for space mission encounters by providing data over longer timescales, at different wavelengths and using techniques that are impossible with an in situ flyby. We report here such observations in support of the EPOXI spacecraft flyby of comet 103P/Hartley 2. The nucleus is small and dark, and exhibited a very rapidly changing rotation period. Prior to the onset of activity, the period was 16.4hr. Starting in 2010 August the period changed from 16.6hr to near 19hr in December. With respect to dust composition, most volatiles and carbon and nitrogen isotope ratios, the comet is similar to other Jupiter-family comets. What is unusual is the dominance of CO 2 -driven activity near perihelion, which likely persists out to aphelion. Near perihelion the comet nucleus was surrounded by a large halo of water-ice grains that contributed significantly to the total water production. © 2011. The American Astronomical Society. All rights reserved..

We present multi-waveband optical imaging data obtained from observations of the Subaru/XMM-Newton Deep Survey (SXDS). The survey field, centered at R.A.=02:18:00, decl.=-05:00:00, has been the focus of a wide range of multi-wavelength observing programs spanning from X-ray to radio wavelengths. A large part of the optical imaging observations are carried out with Suprime-Cam on Subaru Telescope at Mauna Kea in the course of Subaru Telescope Observatory Projects. This paper describes our optical observations, data reduction and analysis procedures employed, and the characteristics of the data products. A total area of 1.22 sqdeg is covered in five contiguous sub-fields, each of which corresponds to a single Suprime-Cam field of view (34'x27'), in five broad-band filters B, V, Rc, i', z' to the depths of B=28.4, V=27.8, Rc=27.7, i'=27.7 and z'=26.6 (AB, 3-sigma, 2-arcsec aperture). The data are reduced and compiled into five multi-waveband photometric catalogs, separately for each Suprime-Cam pointing. The i'-band catalogs contain about 900,000 objects, making the SXDS catalogs one of the largest multi-waveband catalogs in corresponding depth and area coverage. The SXDS catalogs can be used for an extensive range of astronomical applications such as the number density of the Galactic halo stars to the large scale structures at the distant universe. The number counts of galaxies are derived and compared with those of existing deep extragalactic surveys. The optical data, the source catalogs, and configuration files used to create the catalogs are publicly available via the SXDS web page (http://www.naoj.org/Science/SubaruProject/SXDS/index.html), 55 pages, 22 figures, 9 tables, accepted by ApJS, a higher-resolution version is available at http://step.mtk.nao.ac.jp/sxds/

We investigate the Hubbard–Holstein model by using the dynamical mean-field theory combined with the exact diagonalization method. Below a critical temperature Tcr, a coexistence of two solutions, small lattice fluctuating solution (polaronic state) and large one (bipolaronic state), is found for the same value of the electron–phonon coupling g in the range gc1Tp, while it is single-well type for T

25th Workshop on Radiation Detectors and Their Uses (February 1-3, 2011. High Energy Accelerator Research Organization (KEK)), Tsukuba-shi, Ibaraki, Japan, 資料番号: PA1110054000

Automatic photogrammetric processing of historical (or archival) aerial photos is still a challenging task, particularly in cases of missing ancillary information, low radiometric and image quality, limited stereo coverage or large temporal span. However, with recent advances in photogrammetry and Artificial Intelligence (AI) algorithms for image processing and interpretation, an increasing number of applications are now feasible. The article presents the TIME (hisTorical aerIal iMagEs) benchmark (https://time.fbk.eu/), promoted by EuroSDR to explore the potential of historical aerial images. Realized in collaboration with various European NMCAs, the benchmark has garnered aerial image blocks and time series imagery captured since the 1950s. To support the photogrammetric processing of the digitized photos, ancillary data are supplied with available information about flight missions, taking cameras, and ground control points (GCPs). Several diverse investigations have been undertaken with the benchmark datasets, all captured over historical urban areas or landscapes. The paper describes the benchmark datasets and some potential research topics, presenting several tests and analyses realized with the collated and shared data.

Classification and segmentation of buildings from airborne lidar point clouds commonly involve point features calculated within a local neighborhood. The relative change of the features in the immediate surrounding of each point as well as the spatial relationships between neighboring points also need to be examined to account for spatial coherence. In this study we formulate the point labeling problem under a global graph-cut optimization solution. We construct the energy function through a graph representing a Markov Random Field (MRF). The solution to the labeling problem is obtained by finding the minimum-cut on this graph. We have employed this framework for three different labeling tasks on airborne lidar point clouds. Ground filtering, building classification, and roof-plane segmentation. As a follow-up study on our previous ground filtering work, this paper examines our building extraction approach on two airborne lidar datasets with different point densities containing approximately 930K points in one dataset and 750K points in the other. Test results for building vs. non-building point labeling show a 97.9% overall accuracy with a kappa value of 0.91 for the dataset with 1.18 pts/m2 average point density and a 96.8% accuracy with a kappa value of 0.90 for the dataset with 8.83 pts/m2 average point density. We can achieve 91.2% overall average accuracy in roof plane segmentation with respect to the reference segmentation of 20 building roofs involving 74 individual roof planes. In summary, the presented framework can successfully label points in airborne lidar point clouds with different characteristics for all three labeling problems we have introduced. It is robust to noise in the calculated features due to the use of global optimization. Furthermore, the framework achieves these results with a small training sample size. © 2020 Copernicus GmbH. All rights reserved., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, V, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

Traditional flow velocity measurements in natural environments require contact with the fluid and are usually costly, time-consuming and, sometimes, even dangerous. Particle Image Velocimetry allows the flow velocity field to be remotely characterized from the shift of intensity patterns of sub-image areas in at least two video frames with a known time lag. Recently, Airborne Image Velocimetry has enabled the surface velocity field of large-scale water bodies to be determined by applying Particle Image Velocimetry on videos recorded by cameras mounted on unmanned aerial vehicles. This work presents a comparison of three Airborne Image Velocimetry approaches: BASESURV, Fudaa-LSPIV and RIVeR. For the evaluation, two nadiral videos were acquired with a low-cost quadcopter. The first was recorded under low flow and seeded conditions, the second during a flood event. According to the results obtained, BASESURV is an accurate and complete research oriented approach but it is time-consuming and neither a graphical interface nor documentation are yet provided. Fudaa-LSPIV is a well-developed software package, with a user-friendly graphical interface and good documentation. However it lacks some features and the source code is closed. RIVeR may be suitable for real time monitoring thanks to the rectification of velocity vectors only. Overall, all the codes are found to be effective in performing Airborne Image Velocimetry in riverine environments. © 2020 International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLIII, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

Hyperspectral image classification has drawn significant attention in the recent years driven by the increasing abundance of sensor-generated hyper- and multi-spectral data, combined with the rapid advancements in the field of machine learning. A vast range of techniques, especially involving deep learning models, have been proposed attaining high levels of classification accuracy. However, many of these approaches significantly deteriorate in performance in the presence of noise in the hyperspectral data. In this paper, we propose a new model that effectively addresses the challenge of noise residing in hyperspectral images. The proposed model, which will be called DCT-CNN, combines the representational power of Convolutional Neural Networks with the noise elimination capabilities introduced by frequency-domain filtering enabled through the Discrete Cosine Transform. In particular, the proposed method entails the transformation of pixel macroblocks to the frequency domain and the discarding of information that corresponds to the higher frequencies in every patch, in which pixel information of abrupt changes and noise often resides. Experiment results in Indian Pines, Salinas and Pavia University datasets indicate that the proposed DCT-CNN constitutes a promising new model for accurate hyperspectral image classification offering robustness to different types of noise, such as Gaussian and salt and pepper noise. © 2020 International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS Archives., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLIII, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

Remotely sensed hyperspectral data has widely been used to determine water quality parameters in oceanic waters. However in freshwater basins the dependence between the hyperspectral data and the parameters is more complicated. In this work some ideas are presented concerning the study of this dependence. The data used in this study were collected from the lake Hiidenvesi in southern Finland. The hyperspectral data consists of reflectances in 36 bands in the wavelength area 508…878 nm and the separately measured water quality parameters are turbidity, blue-green algae, chlorophyll, pH and dissolved oxygen. Hyperspectral data was used as bare band reflectances, but also in the form of two simple spectral indices: ratio A / B and difference A − B, where A and B go through all the bands. The correlations of the indices with the parameters were presented visually as 1- or 2-dimensional arrays. To examine the significance on the results of different variables, the data was classified in two different ways: the natural basins and the values of the water quality parameters. It was noticed that the variability of the correlation arrays was particularly strong among different basins in both the magnitude of correlation and the best performing indices. Further studies are needed to clarify which features of the basins are of most importance in predicting the shapes of the correlation arrays.

The advantages of terrestrial laser scanning (TLS) for geodetic monitoring of man-made and natural objects are not yet fully exploited. Herein we address one of the open challenges by proposing feature-based methods for identification of corresponding points in point clouds of two or more epochs. We propose a learned compact feature descriptor tailored for point clouds of natural outdoor scenes obtained using TLS. We evaluate our method both on a benchmark data set and on a specially acquired outdoor dataset resembling a simplified monitoring scenario where we successfully estimate 3D displacement vectors of a rock that has been displaced between the scans. We show that the proposed descriptor has the capacity to generalize to unseen data and achieves state-of-the-art performance while being time efficient at the matching step due the low dimension., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, IV-2, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

We propose to use a discriminative classifier for outlier detection in large-scale point clouds of cities generated via multi-view stereo (MVS) from densely acquired images. What makes outlier removal hard are varying distributions of inliers and outliers across a scene. Heuristic outlier removal using a specific feature that encodes point distribution often delivers unsatisfying results. Although most outliers can be identified correctly (high recall), many inliers are erroneously removed (low precision), too. This aggravates object 3D reconstruction due to missing data. We thus propose to discriminatively learn class-specific distributions directly from the data to achieve high precision. We apply a standard Random Forest classifier that infers a binary label (inlier or outlier) for each 3D point in the raw, unfiltered point cloud and test two approaches for training. In the first, non-semantic approach, features are extracted without considering the semantic interpretation of the 3D points. The trained model approximates the average distribution of inliers and outliers across all semantic classes. Second, semantic interpretation is incorporated into the learning process, i.e. we train separate inlieroutlier classifiers per semantic class (building facades, roof, ground, vegetation, fields, and water). Performance of learned filtering is evaluated on several large SfM point clouds of cities. We find that results confirm our underlying assumption that discriminatively learning inlier-outlier distributions does improve precision over global heuristics by up to ≈ 12 percent points. Moreover, semantically informed filtering that models class-specific distributions further improves precision by up to ≈ 10 percent points, being able to remove very isolated building, roof, and water points while preserving inliers on building facades and vegetation., ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, IV-2, ISSN:2194-9042

Estimation of rock breakage characteristics plays an important role in optimising various industrial and mining processes used for rock comminution. Although little research has been undertaken into 3D photogrammetric measurement of the progeny kinematics, there is promising potential to improve the efficacy of rock breakage characterisation. In this study, the observation of progeny kinematics was conducted using a high speed, stereo videometric system based on laboratory experiments with a drop weight impact testing system. By manually tracking individual progeny through the captured video sequences, observed progeny coordinates can be used to determine 3D trajectories and velocities, supporting the idea that high speed video can be used for rock breakage characterisation purposes. An analysis of the results showed that the high speed videometric system successfully observed progeny trajectories and showed clear projection of the progeny away from the impact location. Velocities of the progeny could also be determined based on the trajectories and the video frame rate. These results were obtained despite the limitations of the photogrammetric system and experiment processes observed in this study. Accordingly there is sufficient evidence to conclude that high speed videometric systems are capable of observing progeny kinematics from drop weight impact tests. With further optimisation of the systems and processes used, there is potential for improving the efficacy of rock breakage characterisation from measurements with high speed videometric systems., ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, IV-2, ISSN:2194-9042, ISSN:2194-9050

The paper presents an investigation about the combination of multispectral and 3D imaging aiming at the analysis of the condition and preservation of an ancient vase. Visible-reflected (VIS) and -induced luminescence (UVL) images are mapped to 3D models produced with image- and range-based 3D modelling techniques. The case study is an Attic vase, part of the pottery collection of the Landesmuseum Rudolfinum (Carinthia, Austria) and temporarily stored in the Institute of Archaeology of the University of Graz, Austria. The aim of this study is to exploit the added-value provided by mapping multispectral imaging onto 3D geometry for a comprehensive knowledge of the condition of a restored Cultural Heritage (CH) item., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

An underwater imaging system with camera and lens behind a flat port does not behave as a standard pinhole camera with additional parameters. Indeed, whenever the entrance pupil of the lens is not in contact with the flat port, the standard photogrammetric model is not suited anymore and an extended mathematical model that considers the different media would be required. Therefore, when dealing with flat ports, the use of the classic photogrammetric formulation represents a simplification of the image formation phenomenon, clearly causing a degradation in accuracy. Furthermore, flat ports significantly change the characteristics of the enclosed imaging device and negatively affect the image quality, introducing heavy curvilinear distortions and optical aberrations. With the aim of mitigating the effect of systematic errors introduced by a combination of (i) image quality degradation, induced by the flat ports, and (ii) a non-rigorous modelling of refraction, this paper presents a stochastic model for image observations that penalises those that are more affected by aberrations and departure from the pinhole model. Experiments were carried out at sea and in pools showing that the use of the proposed stochastic model is beneficial for the final accuracy with improvements up to 50 %.

Continuous monitoring of climate indicators is important for understanding the dynamics and trends of the climate system. Lake ice has been identified as one such indicator, and has been included in the list of Essential Climate Variables (ECVs). Currently there are two main ways to survey lake ice cover and its change over time, in-situ measurements and satellite remote sensing. The challenge with both of them is to ensure sufficient spatial and temporal resolution. Here, we investigate the possibility to monitor lake ice with video streams acquired by publicly available webcams. Main advantages of webcams are their high temporal frequency and dense spatial sampling. By contrast, they have low spectral resolution and limited image quality. Moreover, the uncontrolled radiometry and low, oblique viewpoints result in heavily varying appearance of water, ice and snow. We present a workflow for pixel-wise semantic segmentation of images into these classes, based on state-of-the-art encoder-decoder Convolutional Neural Networks (CNNs). The proposed segmentation pipeline is evaluated on two sequences featuring different ground sampling distances. The experiment suggests that (networks of) webcams have great potential for lake ice monitoring. The overall per-pixel accuracies for both tested data sets exceed 95 %. Furthermore, per-image discrimination between ice-on and ice-off conditions, derived by accumulating per-pixel results, is 100 % correct for our test data, making it possible to precisely recover freezing and thawing dates., ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, IV-2, ISSN:2194-9042, ISSN:2194-9050

Ecological studies about marine benthic communities received a major leap from the application of a variety of non-destructive sampling and mapping techniques based on underwater image and video recording. The well-established scientific diving practice consists in the acquisition of single path or ‘round-trip’ over elongated transects, with the imaging device oriented in a nadir looking direction. As it may be expected, the application of automatic image processing procedures to data not specifically acquired for 3D modelling can be risky, especially if proper tools for assessing the quality of the produced results are not employed. This paper, born from an international cooperation, focuses on this topic, which is of great interest for ecological and monitoring benthic studies in Antarctica. Several video footages recorded from different scientific teams in different years are processed with an automatic photogrammetric procedure and salient statistical features are reported to critically analyse the derived results. As expected, the inclusion of oblique images from additional lateral strips may improve the expected accuracy in the object space, without altering too much the current video recording practices., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLII-2, ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

Creating 3-dimensional (3D) models of underwater scenes has become a common approach for monitoring coral reef changes and its structural complexity. Also in underwater archeology, 3D models are often created using underwater optical imagery. In this paper, we focus on the aspect of detecting small changes in the coral reef using a multi-temporal photogrammetric modelling approach, which requires a high quality control network. We show that the quality of a good geodetic network limits the direct change detection, i.e., without any further registration process. As the photogrammetric accuracy is expected to exceed the geodetic network accuracy by at least one order of magnitude, we suggest to do a fine registration based on a number of signalized points. This work is part of the Moorea Island Digital Ecosystem Avatar (IDEA) project that has been initiated in 2013 by a group of international researchers (https://mooreaidea.ethz.ch/)., International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 42 (2), ISSN:1682-1750, ISSN:2194-9034, ISSN:1682-1777

Accurate 3D data is of high importance for indoor modeling for various applications in construction, engineering and cultural heritage documentation. For the lack of GNSS signals hampers use of kinematic platforms indoors, TLS is currently the most accurate and precise method for collecting such a data. Due to its static single view point data collection, excessive time and data redundancy are needed for integrity and coverage of data. However, localization methods with affordable scanners are used for solving mobile platform pose problem. The aim of this study was to investigate what level of trajectory accuracies can be achieved with high quality sensors and freely available state of the art planar SLAM algorithms, and how well this trajectory translates to a point cloud collected with a secondary scanner. In this study high precision laser scanners were used with a novel way to combine the strengths of two SLAM algorithms into functional method for precise localization. We collected five datasets using Slammer platform with two laser scanners, and processed them with altogether 20 different parameter sets. The results were validated against TLS reference. The results show increasing scan frequency improves the trajectory, reaching 20 mm RMSE levels for the best performing parameter sets. Further analysis of the 3D point cloud showed good agreement with TLS reference with 17 mm positional RMSE. With precision scanners the obtained point cloud allows for high level of detail data for indoor modeling with accuracies close to TLS at best with vastly improved data collection efficiency.

This paper investigates the use of different greyscale conversion algorithms to decolourize colour images as input for two Structure-from-Motion (SfM) software packages. Although SfM software commonly works with a wide variety of frame imagery (old and new, colour and greyscale, airborne and terrestrial, large-and small scale), most programs internally convert the source imagery to single-band, greyscale images. This conversion is often assumed to have little, if any, impact on the final outcome. To verify this assumption, this article compares the output of an academic and a commercial SfM software package using seven different collections of architectural images. Besides the conventional 8-bit true-colour JPEG images with embedded sRGB colour profiles, for each of those datasets, 57 greyscale variants were computed with different colour-to-greyscale algorithms. The success rate of specific colour conversion approaches can therefore be compared with the commonly implemented colour-to-greyscale algorithms (luma Y’601, luma Y’709, or luminance CIE Y), both in terms of the applied feature extractor as well as of the specific image content (as exemplified by the two different feature descriptors and the various image collections, respectively). Although the differences can be small, the results clearly indicate that certain colour-to-greyscale conversion algorithms in an SfM-workflow constantly perform better than others. Overall, one of the best performing decolourization algorithms turns out to be a newly developed one.

Individual archaeological sites can sometimes show a complex morphology. One such site is the Roman quarries located one kilometre northwest of the Roman fortress at Udhruh, in the hinterland of Petra. In archaeology there are various platforms such as balloons, kites and unmanned aerial vehicles (uav's) from which low altitude aerial photographs can be taken that have been proven to work. All with their specific advantages above others in different circumstances. By designing a very distinct setup for the kite and optimizing the workflow accordingly, an effective and steady platform for the camera was created. The Quarries were photographed from the air and the images then provided enough material to accurately create a 3D model of the site.