Search

3D Gaussian Splatting (3DGS) has shown significant advantages in novel view synthesis (NVS), particularly in achieving high rendering speeds and high-quality results. However, its geometric accuracy in 3D reconstruction remains limited due to the lack of explicit geometric constraints during optimization. This paper introduces CDGS, a confidence-aware depth regularization approach developed to enhance 3DGS. We leverage multi-cue confidence map of monocular depth estimation and sparse Structure-from-Motion (SfM) depth to adaptively adjusts depth supervision during the optimization process. Our method demonstrates improved geometric detail preservation in early training stages and achieves competitive performance in both NVS quality and geometric accuracy. Experiments on the public available Tanks and Temples benchmark dataset show that our method achieves more stable convergence behavior and more accurate geometric reconstruction results, with improvements of up to 2.31 dB in PSNR for NVS and consistently lower geometric errors in M3C2 distance metrics. Notably, our method reaches comparable F-scores to the original 3DGS with only 50% of the training iterations. We expect this work will facilitate the development of efficient and accurate 3D reconstruction systems for real-world applications such as digital twin creation, heritage preservation, or forestry applications.

Super resolution for satellite imagery possesses specific challenges due to its unique feature geometry compared to classic computer vision. Specifically, satellite images contain a high amount of relatively small, distributed high-frequency features that are hard to preserve when sampling up to a higher resolution. General adversarial networks (GANs) are suitable for super resolution tasks but need special attention concerning the geometric and radiometric accuracy of the results. We propose a network for versatile satellite imagery super resolution (VSISR) that focuses on high-frequency detail preservation and radiometric consistency. As a novelty, it is able to utilize a reference high resolution image during inference to lower hallucinations without altering the source image’s radiometric characteristics.A GAN that is already handling well high frequencies in standard computer vision cases is adapted for satellite imagery and supplemented with a mixed pixel approach for data augmentation. Training on a diverse RGB dataset from four satellite missions results in a versatile super resolution model that is optimized to preserve radiometric features and minimize hallucinations. Compared to other neural networks for satellite image super resolution in their respective datasets, VSISR performs in the middle regarding mean PSNR (25.30 dB) and SSIM (0.8098). Nevertheless, it is the first time that, using mixed pixel training on a comparably small dataset, this versatility concerning the satellite data source is achieved while maintaining their unique radiometric traits.

In the fields of photogrammetry, computer vision and computer graphics, the task of neural 3D scene reconstruction has led to the exploration of various techniques. Among these, 3D Gaussian Splatting stands out for its explicit representation of scenes using 3D Gaussians, making it appealing for tasks like 3D point cloud extraction and surface reconstruction. Motivated by its potential, we address the domain of 3D scene reconstruction, aiming to leverage the capabilities of the Microsoft HoloLens 2 for instant 3D Gaussian Splatting. We present HoloGS, a novel workflow utilizing HoloLens sensor data, which bypasses the need for pre-processing steps like Structure from Motion by instantly accessing the required input data i.e. the images, camera poses and the point cloud from depth sensing. We provide comprehensive investigations, including the training process and the rendering quality, assessed through the Peak Signal-to-Noise Ratio, and the geometric 3D accuracy of the densified point cloud from Gaussian centers, measured by Chamfer Distance. We evaluate our approach on two self-captured scenes: An outdoor scene of a cultural heritage statue and an indoor scene of a fine-structured plant. Our results show that the HoloLens data, including RGB images, corresponding camera poses, and depth sensing based point clouds to initialize the Gaussians, are suitable as input for 3D Gaussian Splatting.

Generating geometric 3D reconstructions from Neural Radiance Fields (NeRFs) is of great interest. However, accurate and complete reconstructions based on the density values are challenging. The network output depends on input data, NeRF network configuration and hyperparameter. As a result, the direct usage of density values, e.g. via filtering with global density thresholds, usually requires empirical investigations. Under the assumption that the density increases from non-object to object area, the utilization of density gradients from relative values is evident. As the density represents a position-dependent parameter it can be handled anisotropically, therefore processing of the voxelized 3D density field is justified. In this regard, we address geometric 3D reconstructions based on density gradients, whereas the gradients result from 3D edge detection filters of the first and second derivatives, namely Sobel, Canny and Laplacian of Gaussian. The gradients rely on relative neighboring density values in all directions, thus are independent from absolute magnitudes. Consequently, gradient filters are able to extract edges along a wide density range, almost independent from assumptions and empirical investigations. Our approach demonstrates the capability to achieve geometric 3D reconstructions with high geometric accuracy on object surfaces and remarkable object completeness. Notably, Canny filter effectively eliminates gaps, delivers a uniform point density, and strikes a favorable balance between correctness and completeness across the scenes.

Neural Radiance Fields (NeRFs) use a set of camera poses with associated images to represent a scene through a position-dependent density and radiance at given spatial location. Generating a geometric representation in form of a point cloud is gained by ray tracing and sampling 3D points with density and color along the rays. In this contribution we evaluate object reconstruction by NeRFs in 3D metric space against Terrestrial Laser Scanning (TLS) using ground truth data in form of a Structured Light Imaging (SLI) mesh and investigate the influence of the density to the reconstruction’s accuracy. We extend the accuracy assessment from 2D to 3D space and perform high resolution investigations on NeRFs by using camera images with 36MP resolution as well as comparison among point clouds of more than 20 million points against a 0.1mm ground truth mesh. TLS achieves the highest geometric accuracy results with a standard deviation of 1.68mm, while NeRFδt=300 diverges 18.61mm from the ground truth. All NeRF reconstructions contain 3D points inside the object which have the highest displacements from the ground truth, thus contribute the most to the accuracy results. NeRFs accuracy improves with increasing the density threshold as a consequence of completeness, since beside noise and outliers the object points are also being removed.

This work represents a large step into modern ways of fast 3D reconstruction based on RGB camera images. Utilizing a Microsoft HoloLens 2 as a multisensor platform that includes an RGB camera and an inertial measurement unit for SLAM-based camera-pose determination, we train a Neural Radiance Field (NeRF) as a neural scene representation in real-time with the acquired data from the HoloLens. The HoloLens is connected via Wifi to a high-performance PC that is responsible for the training and 3D reconstruction. After the data stream ends, the training is stopped and the 3D reconstruction is initiated, which extracts a point cloud of the scene. With our specialized inference algorithm, five million scene points can be extracted within 1 second. In addition, the point cloud also includes radiometry per point. Our method of 3D reconstruction outperforms grid point sampling with NeRFs by multiple orders of magnitude and can be regarded as a complete real-time 3D reconstruction method in a mobile mapping setup.

Neural Radiance Fields (NeRFs) are trained using a set of camera poses and associated images as input to estimate density and color values for each position. The position-dependent density learning is of particular interest for photogrammetry, enabling 3D reconstruction by querying and filtering the NeRF coordinate system based on the object density. While traditional methods like Structure from Motion are commonly used for camera pose calculation in pre-processing for NeRFs, the HoloLens offers an interesting interface for extracting the required input data directly. We present a workflow for high-resolution 3D reconstructions almost directly from HoloLens data using NeRFs. Thereby, different investigations are considered: Internal camera poses from the HoloLens trajectory via a server application, and external camera poses from Structure from Motion, both with an enhanced variant applied through pose refinement. Results show that the internal camera poses lead to NeRF convergence with a PSNR of 25 dB with a simple rotation around the x-axis and enable a 3D reconstruction. Pose refinement enables comparable quality compared to external camera poses, resulting in improved training process with a PSNR of 27 dB and a better 3D reconstruction. Overall, NeRF reconstructions outperform the conventional photogrammetric dense reconstruction using Multi-View Stereo in terms of completeness and level of detail.

Corrosion is a form of damage that often appears on the surface of metal-made objects used in industrial applications. Those damages can be critical depending on the purpose of the used object. Optical-based testing systems provide a form of non-contact data acquisition, where the acquired data can then be used to analyse the surface of an object. In the field of industrial image processing, this is called surface inspection. We provide a testing setup consisting of a rotary table which rotates the object by 360 degrees, as well as industrial RGB cameras and laser triangulation sensors for the acquisition of 2D and 3D data as our multi-sensor system. These sensors acquire data while the object to be tested takes a full rotation. Further on, data augmentation is applied to prepare new data or enhance already acquired data. In order to evaluate the impact of a laser triangulation sensor for corrosion detection, one challenge is to at first fuse the data of both domains. After the data fusion process, 5 different channels can be utilized to create a 5D feature space. Besides the red, green and blue channels of the image (1–3), additional range data from the laser triangulation sensor is incorporated (4). As a fifth channel, said sensor provides additional intensity data (5). With a multi-channel image classification, a 5D feature space will lead to slightly superior results opposed to a 3D feature space, composed of only the RGB channels of the image.

The field of Earth Observation (EO) and Geoinformation (GI) is gaining more and more importance due to the increasing number of data and data processing algorithms to respond even more accurately to a variety of challenges in many application areas. In order to follow recent activities and align the exponential evolution of datasets and recent processing trends with market and academic training requirements, the EO*GI sector needs an updated and new definition of knowledge and skills. To this goal, a specific body of knowledge for Earth Observation and Geoinformation (EO4GEO BoK) is currently being implemented with the aim of providing interconnected concepts and job-oriented skills. One novelty of the BoK is to include topics related to Earth Observation, in particular to introduce the different sensors used in this field. Active optical sensors are at the crossroad between Earth Observation and close-range photogrammetry and have not been described in any other existing BoK. This paper introduces therefore the part of the EO4GEO BoK that is dedicated to active optical sensors. Such systems are used in various job-oriented applications such as archeology, mobile mapping, indoor or outdoor, or for surveying purposes, just to name a few. The structure of this part of the BoK is explained and specific descriptions and relationships between the identified concepts are given. Finally, the skills acquired by completing these BoK concepts are presented and discussed in terms of how they can be used in a professional context or in the definition of job-oriented learning paths.

3D indoor mapping and scene understanding have seen tremendous progress in recent years due to the rapid development of sensor systems, reconstruction techniques and semantic segmentation approaches. However, the quality of the acquired data strongly influences the accuracy of both reconstruction and segmentation. In this paper, we direct our attention to the evaluation of the mapping capabilities of the Microsoft HoloLens in comparison to high-quality TLS systems with respect to 3D indoor mapping, feature extraction and semantic segmentation. We demonstrate how a set of rather interpretable low-level geometric features and the resulting semantic segmentation achieved with a Random Forest classifier applied on these features are affected by the quality of the acquired data. The achieved results indicate that, while allowing for a fast acquisition of room geometries, the HoloLens provides data with sufficient accuracy for a wide range of applications.

The extrinsic calibration of a Mobile Laser Scanning system aims to determine the relative orientation between a laser scanner and a sensor that estimates the exterior orientation of the sensor system. The relative orientation is one component that limits the accuracy of a 3D point cloud which is captured with a Mobile Laser Scanning system. The most efficient way to determine the relative orientation of a Mobile Laser Scanning system is using a self-calibration approach as this avoids the need to perform an additional calibration beforehand. Instead, the system can be calibrated automatically during data acquisition. The entropy-based self-calibration fits into this category and is utilized in this contribution. In this contribution, we analyze the impact of four different trajectories on the result of the entropy-based self-calibration, namely (i) uni-directional, (ii) ortho-directional, (iii) bi-directional, and (iv) multi-directional trajectory. Theoretical considerations are supported by experiments performed with the publicly available MLS 1 – TUM City Campus data set. The investigations show that strong variations of the yaw angle in a confined space or bidirectional trajectories as well as the variation of the height of the laser scanner are beneficial for calibration.

The performance of machine learning and deep learning algorithms for image analysis depends significantly on the quantity and quality of the training data. The generation of annotated training data is often costly, time-consuming and laborious. Data augmentation is a powerful option to overcome these drawbacks. Therefore, we augment training data by rendering images with arbitrary poses from 3D models to increase the quantity of training images. These training images usually show artifacts and are of limited use for advanced image analysis. Therefore, we propose to use image-to-image translation to transform images from a rendered domain to a captured domain. We show that translated images in the captured domain are of higher quality than the rendered images. Moreover, we demonstrate that image-to-image translation based on rendered 3D models enhances the performance of common computer vision tasks, namely feature matching, image retrieval and visual localization. The experimental results clearly show the enhancement on translated images over rendered images for all investigated tasks. In addition to this, we present the advantages utilizing translated images over exclusively captured images for visual localization.

Image-based localization or camera re-localization is a fundamental task in computer vision and mandatory in the fields of navigation for robotics and autonomous driving or for virtual and augmented reality. Such image pose regression in 6 Degrees of Freedom (DoF) is recently solved by Convolutional Neural Networks (CNNs). However, already well-established methods based on feature matching still score higher accuracies so far. Therefore, we want to investigate how data augmentation could further improve CNN-based pose regression. Data augmentation is a valuable technique to boost performance on training based methods and wide spread in the computer vision community. Our aim in this paper is to show the benefit of data augmentation for pose regression by CNNs. For this purpose images are rendered from a 3D model of the actual test environment. This model again is generated by the original training data set, whereas no additional information nor data is required. Furthermore we introduce different training sets composed of rendered and real images. It is shown that the enhanced training of CNNs by utilizing 3D models of the environment improves the image localization accuracy. The accuracy of pose regression could be improved up to 69.37 % for the position component and 61.61 % for the rotation component on our investigated data set.

The recent advent of single photon sensitive airborne LiDAR (Light Detection And Ranging) sensors has enabled higher areal coverage performance at the price of an increased outlier rate and a lower ranging accuracy compared to conventional Multi-Photon LiDAR. Single Photon LiDAR, in particular, uses green laser light capable of penetrating clear shallow water. Although primarily designed for large area topographic mapping, the technique can also be used for mapping the water surface and shallow water bathymetry. In this contribution we investigate the capability of Single Photon LiDAR for large area mapping of water surface heights. While interface returns from conventional green-only bathymetric sensors generally suffer from water level underestimation due to the water penetration capabilities of green laser radiation, the specific questions are, if Single Photon LiDAR (i) is less affected by this well known effect due to the high receiver sensitivity and (ii) consequently delivers a higher number of water surface echoes. The topic is addressed empirically in a case study by comparing the water surface responses of Single Photon LiDAR (Navarra, Spain) and Multi-Photon Topo-Bathymetric LiDAR (Neubacher Au, Austria) for selected water bodies with a horizontal water surface (reservoirs, ponds). Although flown at different altitudes, both datasets are well comparable as they exhibit the same strip point density of ca. 14 points/m2. The expected superiority of Single Photon LiDAR over conventional green-only bathymetric LiDAR for mapping water surfaces could not be verified in this investigation. While both datasets show good agreement compared to a reference water level when aggregating points into cells of 10 × 10 m2 (mean deviations

In this paper, we address the semantic segmentation of aerial imagery based on the use of multi-modal data given in the form of true orthophotos and the corresponding Digital Surface Models (DSMs). We present the Deeply-supervised Shuffling Convolutional Neural Network (DSCNN) representing a multi-scale extension of the Shuffling Convolutional Neural Network (SCNN) with deep supervision. Thereby, we take the advantage of the SCNN involving the shuffling operator to effectively upsample feature maps and then fuse multiscale features derived from the intermediate layers of the SCNN, which results in the Multi-scale Shuffling Convolutional Neural Network (MSCNN). Based on the MSCNN, we derive the DSCNN by introducing additional losses into the intermediate layers of the MSCNN. In addition, we investigate the impact of using different sets of hand-crafted radiometric and geometric features derived from the true orthophotos and the DSMs on the semantic segmentation task. For performance evaluation, we use a commonly used benchmark dataset. The achieved results reveal that both multi-scale fusion and deep supervision contribute to an improvement in performance. Furthermore, the use of a diversity of hand-crafted radiometric and geometric features as input for the DSCNN does not provide the best numerical results, but smoother and improved detections for several objects.

In this paper, we address the deep semantic segmentation of aerial imagery based on multi-modal data. Given multi-modal data composed of true orthophotos and the corresponding Digital Surface Models (DSMs), we extract a variety of hand-crafted radiometric and geometric features which are provided separately and in different combinations as input to a modern deep learning framework. The latter is represented by a Residual Shuffling Convolutional Neural Network (RSCNN) combining the characteristics of a Residual Network with the advantages of atrous convolution and a shuffling operator to achieve a dense semantic labeling. Via performance evaluation on a benchmark dataset, we analyze the value of different feature sets for the semantic segmentation task. The derived results reveal that the use of radiometric features yields better classification results than the use of geometric features for the considered dataset. Furthermore, the consideration of data on both modalities leads to an improvement of the classification results. However, the derived results also indicate that the use of all defined features is less favorable than the use of selected features. Consequently, data representations derived via feature extraction and feature selection techniques still provide a gain if used as the basis for deep semantic segmentation.

The basic requirement for the successful deployment of a mobile augmented reality application is a reliable tracking system with high accuracy. Recently, a helmet-based inside-out tracking system which meets this demand has been proposed for self-localization in buildings. To realize an augmented reality application based on this tracking system, a display has to be added for visualization purposes. Therefore, the relative pose of this visualization platform with respect to the helmet has to be tracked. In the case of hand-held visualization platforms like smartphones or tablets, this can be achieved by means of image-based tracking methods like marker-based or model-based tracking. In this paper, we present two marker-based methods for tracking the relative pose between the helmet-based tracking system and a tablet-based visualization system. Both methods were implemented and comparatively evaluated in terms of tracking accuracy. Our results show that mobile inside-out tracking systems without integrated displays can easily be supplemented with a hand-held tablet as visualization device for augmented reality purposes.

Classification of materials found in urban areas using remote sensing, in particular with hyperspectral data, has in recent times increased in importance. This study is conducting classification of materials found on building using hyperspectral data, by using an existing spectral library and collected data acquired with a spectrometer. Two commonly used classification algorithms, Support Vector Machine and Random Forest, were used to classify the materials. In addition, dimensionality reduction and band selection were performed to determine if selected parts of the full spectral domain, such as the Short Wave Infra-Red domain, are sufficient to classify the different materials. We achieved the best classification results for the two datasets using dimensionality reduction based on a Principal Component Analysis in combination with a Random Forest classification. Classification using the full domain achieved the best results, followed by the Short Wave Infra-Red domain.

In this paper, we focus on UAV-borne laser scanning with the objective of densely sampling object surfaces in the local surrounding of the UAV. In this regard, using a line scanner which scans along the vertical direction and perpendicular to the flight direction results in a point cloud with low point density if the UAV moves fast. Using a line scanner which scans along the horizontal direction only delivers data corresponding to the altitude of the UAV and thus a low scene coverage. For these reasons, we present a concept and a system for UAV-borne laser scanning using multiple line scanners. Our system consists of a quadcopter equipped with horizontally and vertically oriented line scanners. We demonstrate the capabilities of our system by presenting first results obtained for a flight within an outdoor scene. Thereby, we use a downsampling of the original point cloud and different neighborhood types to extract fundamental geometric features which in turn can be used for scene interpretation with respect to linear, planar or volumetric structures.

Unmanned Aerial Vehicle (UAV) with adequate sensors enable new applications in the scope between expensive, large-scale, aircraftcarried remote sensing and time-consuming, small-scale, terrestrial surveyings. To perform these applications, cameras and laserscanners are a good sensor combination, due to their complementary properties. To exploit this sensor combination the intrinsics and relative poses of the individual cameras and the relative poses of the cameras and the laserscanners have to be known. In this manuscript, we present a calibration methodology for the Unified Intrinsic and Extrinsic Calibration of a Multi-Camera-System and a Laserscanner (UCalMiCeL). The innovation of this methodology, which is an extension to the calibration of a single camera to a line laserscanner, is an unifying bundle adjustment step to ensure an optimal calibration of the entire sensor system. We use generic camera models, including pinhole, omnidirectional and fisheye cameras. For our approach, the laserscanner and each camera have to share a joint field of view, whereas the fields of view of the individual cameras may be disjoint. The calibration approach is tested with a sensor system consisting of two fisheye cameras and a line laserscanner with a range measuring accuracy of 30 mm. We evaluate the estimated relative poses between the cameras quantitatively by using an additional calibration approach for Multi-Camera-Systems based on control points which are accurately measured by a motion capture system. In the experiments, our novel calibration method achieves a relative pose estimation with a deviation below 1.8° and 6.4 mm.

The number of unmanned aerial vehicles (UAVs) is increasing since low-cost airborne systems are available for a wide range of users. The outdoor navigation of such vehicles is mostly based on global navigation satellite system (GNSS) methods to gain the vehicles trajectory. The drawback of satellite-based navigation are failures caused by occlusions and multi-path interferences. Beside this, local image-based solutions like Simultaneous Localization and Mapping (SLAM) and Visual Odometry (VO) can e.g. be used to support the GNSS solution by closing trajectory gaps but are computationally expensive. However, if the trajectory estimation is interrupted or not available a re-localization is mandatory. In this paper we will provide a novel method for a GNSS-free and fast image-based pose regression in a known area by utilizing a small convolutional neural network (CNN). With on-board processing in mind, we employ a lightweight CNN called SqueezeNet and use transfer learning to adapt the network to pose regression. Our experiments show promising results for GNSS-free and fast localization.

In this paper, we focus on the automatic interpretation of 3D point cloud data in terms of associating a class label to each 3D point. While much effort has recently been spent on this research topic, little attention has been paid to the influencing factors that affect the quality of the derived classification results. For this reason, we investigate fundamental influencing factors making geometric features more or less relevant with respect to the classification task. We present a framework which consists of five components addressing point sampling, neighborhood recovery, feature extraction, classification and feature relevance assessment. To analyze the impact of the main influencing factors which are represented by the given point sampling and the selected neighborhood type, we present the results derived with different configurations of our framework for a commonly used benchmark dataset for which a reference labeling with respect to three structural classes (linear structures, planar structures and volumetric structures) as well as a reference labeling with respect to five semantic classes (Wire, Pole/Trunk, Façade, Ground and Vegetation) is available.

In recent years, there has been a significant improvement in the detection, identification and classification of objects and images using Convolutional Neural Networks. To study the potential of the Convolutional Neural Network, in this paper three approaches are investigated to train classifiers based on Convolutional Neural Networks. These approaches allow Convolutional Neural Networks to be trained on datasets containing only a few hundred training samples, which results in a successful classification. Two of these approaches are based on the concept of transfer learning. In the first approach features, created by a pretrained Convolutional Neural Network, are used for a classification using a support vector machine. In the second approach a pretrained Convolutional Neural Network gets fine-tuned on a different data set. The third approach includes the design and training for flat Convolutional Neural Networks from the scratch. The evaluation of the proposed approaches is based on a data set provided by the IEEE Geoscience and Remote Sensing Society (GRSS) which contains RGB and LiDAR data of an urban area. In this work it is shown that these Convolutional Neural Networks lead to classification results with high accuracy both on RGB and LiDAR data. Features which are derived by RGB data transferred into LiDAR data by transfer learning lead to better results in classification in contrast to RGB data. Using a neural network which contains fewer layers than common neural networks leads to the best classification results. In this framework, it can furthermore be shown that the practical application of LiDAR images results in a better data basis for classification of vehicles than the use of RGB images.

For ISPRS Technical Commission I (TC I), 76 submissions for the 2021 Congress edition of ISPRS Annals and ISPRS Archives were received. This included both full paper and abstract submissions from all over the world. Continuing the success of double blind paper reviewing in preparation of the 2016 Prague congress, the 2018 Karlsruhe symposium as well as the 2020 edition of the ISPRS congress, also this time, TC I organized a strict peer-reviewing process. This included double-blind reviewing for full papers as well as a two-stage evaluation of abstract submissions – first stage based on the submitted abstracts (“conditional acceptance”), and second stage after submission of the respective final papers.For the 2021 edition of the ISPRS congress, we received 32 full paper submissions and 46 abstract submissions, which underlines the trend towards full paper submissions compared to previous TC I events.17 full papers passed the double-blind peer-review process and were accepted for publication in the ISPRS Annals (acceptance rate 53%); none of the papers was reviewed by less than two reviewers; for the majority of full papers, three or four reviews were obtained. Abstract submissions and their respective final papers were reviewed by a team of professionals. In total, 28 papers were accepted to the ISPRS Archives.Strict peer-reviewing relies on many voluntary reviewers spending their time for reading and evaluating the submissions. We would like to express our thanks to our reviewing team – especially since they were again willing to contribute to the review process just after completion of the 2020 edition of the ISPRS congress.The manuscripts in both the ISPRS Annals and Archives cover a broad range of topics related to remote sensing platforms, technologies, systems and related methods and reflect the current trends in algorithmic research and developments in sensing and data acquisition methods. Noteworthy is that numerous contributions were again submitted to Intercommission WGs of TC I with TC II and TC IV, which underlines the continuing trend towards an integral approach to sensors, systems and methods in photogrammetry, remote sensing and mobile mapping.

For ISPRS Technical Commission I (TC I), 76 submissions for the 2021 Congress edition of ISPRS Annals and ISPRS Archives were received. This included both full paper and abstract submissions from all over the world. Continuing the success of double blind paper reviewing in preparation of the 2016 Prague congress, the 2018 Karlsruhe symposium as well as the 2020 edition of the ISPRS congress, also this time, TC I organized a strict peer-reviewing process. This included double-blind reviewing for full papers as well as a two-stage evaluation of abstract submissions – first stage based on the submitted abstracts (“conditional acceptance”), and second stage after submission of the respective final papers.For the 2021 edition of the ISPRS congress, we received 32 full paper submissions and 46 abstract submissions, which underlines the trend towards full paper submissions compared to previous TC I events.17 full papers passed the double-blind peer-review process and were accepted for publication in the ISPRS Annals (acceptance rate 53%); none of the papers was reviewed by less than two reviewers; for the majority of full papers, three or four reviews were obtained. Abstract submissions and their respective final papers were reviewed by a team of professionals. In total, 28 papers were accepted to the ISPRS Archives.Strict peer-reviewing relies on many voluntary reviewers spending their time for reading and evaluating the submissions. We would like to express our thanks to our reviewing team – especially since they were again willing to contribute to the review process just after completion of the 2020 edition of the ISPRS congress.The manuscripts in both the ISPRS Annals and Archives cover a broad range of topics related to remote sensing platforms, technologies, systems and related methods and reflect the current trends in algorithmic research and developments in sensing and data acquisition methods. Noteworthy is that numerous contributions were again submitted to Intercommission WGs of TC I with TC II and TC IV, which underlines the continuing trend towards an integral approach to sensors, systems and methods in photogrammetry, remote sensing and mobile mapping.

This work describes an iterative algorithm for estimating optimal viewpoints, so called next-best-views (NBVs). The goal is to incrementally construct a topological network from the scene during the consecutive acquisition of several views. Our approach is a hybrid method between a surface-based and a volumetric approach with a continuous model space. Hence, a new scan taken from an optimal position should either cover as much as possible from the unknown object surface in one single scan, or densify the existing data and close possible gaps. Based on the point density, we recover the essential and structural information of a scene based on the Growing Neural Gas (GNG) algorithm. From the created graph representation of topological relations, the density of the point cloud at each network node is estimated by approximating the volume of Voronoi cells. The NBV Finder selects a network node as NBV, which has the lowest point density. Our NBV method is self-terminating when all regions reach a predefined minimum point density or the change of the GNG error is zero. For evaluation, we use a Buddha statue with a rather simple surface geometry but still some concave parts and the Stanford Dragon with a more complex object surface containing occluded and concave parts. We demonstrate that our NBV method outperforms a “naive random” approach relying on uniformly distributed sensor positions in terms of efficiency, i.e. our proposed method reaches a desired minimum point density up to 20% faster with less scans.

In this paper, we address the classification of airborne laser scanning data. We present a novel methodology relying on the use of complementary types of geometric features extracted from multiple local neighbourhoods of different scale and type. To demonstrate the performance of our methodology, we present results of a detailed evaluation on a standard benchmark dataset and we show that the consideration of multi-scale, multi-type neighbourhoods as the basis for feature extraction leads to improved classification results in comparison to single-scale neighbourhoods as well as in comparison to multi-scale neighbourhoods of the same type.

For ISPRS Technical Commission I (TC I), a remarkable number of 189 submissions for the 2020 Congress edition of ISPRS Annals and ISPRS Archives was received. This included both full paper and abstract submissions from all over the world. Encouraged by the success of double blind paper reviewing in preparation of the 2016 Prague congress and the 2018 Karlsruhe symposium, also this time, TC I organized a strict peer-reviewing process. This included double-blind reviewing for full papers as well as a two-stage evaluation of abstract submissions – first stage based on the submitted abstracts (“conditional acceptance”), and second stage after submission of the respective final papers.In total, we received 69 full paper submissions and 120 abstract submissions, which indicates a nice trend towards full paper submissions compared to previous TC I events.45 full papers passed the double-blind peer-review process and were accepted for publication in the ISPRS Annals (acceptance rate 64%); usually, three or four reviews were obtained for each paper. Abstract submissions and their respective final papers were reviewed by a team of professionals. In total, 76 papers were accepted to the ISPRS Archives.The manuscripts in both the ISPRS Annals and Archives cover a broad range of topics related to remote sensing platforms, technologies, systems and related methods and reflect the current trends in algorithmic research and developments in sensing and data acquisition methods. Noteworthy is that numerous contributions were submitted to Intercommission WGs of TC I with TC II and TC IV, which underlines the increasing trend towards an integral approach to sensors, systems and methods in photogrammetry, remote sensing and mobile mapping.

For ISPRS Technical Commission I (TC I), a remarkable number of 189 submissionsfor the 2020 Congress edition of ISPRS Annals and ISPRS Archives was received.This included both full paper and abstract submissions from all over the world.Encouraged by the success of double blind paper reviewing in preparation of the2016 Prague congress and the 2018 Karlsruhe symposium, also this time, TC Iorganized a strict peer-reviewing process. This included double-blind reviewing forfull papers as well as a two-stage evaluation of abstract submissions – first stagebased on the submitted abstracts (“conditional acceptance“), and second stage after submission of the respective final papers.In total, we received 69 full paper submissions and 120 abstract submissions, whichindicates a nice trend towards full paper submissions compared to previous TC Ievents.45 full papers passed the double-blind peer-review process and were accepted forpublication in the ISPRS Annals (acceptance rate 64%); usually, three or four reviewswere obtained for each paper. Abstract submissions and their respective final paperswere reviewed by a team of professionals. In total, 76 papers were accepted to theISPRS Archives.The manuscripts in both the ISPRS Annals and Archives cover a broad range oftopics related to remote sensing platforms, technologies, systems and relatedmethods and reflect the current trends in algorithmic research and developments insensing and data acquisition methods. Noteworthy is that numerous contributionswere submitted to Intercommission WGs of TC I with TC II and TC IV, whichunderlines the increasing trend towards an integral approach to sensors, systemsand methods in photogrammetry, remote sensing and mobile mapping.

The faithful 3D reconstruction of urban environments is an important prerequisite for tasks such as city modeling, scene interpretation or urban accessibility analysis. Typically, a dense and accurate 3D reconstruction is acquired with terrestrial laser scanning (TLS) systems by capturing several scans from different locations, and the respective point clouds have to be aligned correctly in a common coordinate frame. In this paper, we present an accurate and robust method for a keypoint-based registration of unordered point clouds via projective scan matching. Thereby, we involve a consistency check which removes unreliable feature correspondences and thus increases the ratio of inlier correspondences which, in turn, leads to a faster convergence of the RANSAC algorithm towards a suitable solution. This consistency check is fully generic and it not only favors geometrically smooth object surfaces, but also those object surfaces with a reasonable incidence angle. We demonstrate the performance of the proposed methodology on a standard TLS benchmark dataset and show that a highly accurate and robust registration may be achieved in a fully automatic manner without using artificial markers.

The fully automated analysis of 3D point clouds is of great importance in photogrammetry, remote sensing and computer vision. For reliably extracting objects such as buildings, road inventory or vegetation, many approaches rely on the results of a point cloud classification, where each 3D point is assigned a respective semantic class label. Such an assignment, in turn, typically involves statistical methods for feature extraction and machine learning. Whereas the different components in the processing workflow have extensively, but separately been investigated in recent years, the respective connection by sharing the results of crucial tasks across all components has not yet been addressed. This connection not only encapsulates the interrelated issues of neighborhood selection and feature extraction, but also the issue of how to involve spatial context in the classification step. In this paper, we present a novel and generic approach for 3D scene analysis which relies on (i) individually optimized 3D neighborhoods for (ii) the extraction of distinctive geometric features and (iii) the contextual classification of point cloud data. For a labeled benchmark dataset, we demonstrate the beneficial impact of involving contextual information in the classification process and that using individual 3D neighborhoods of optimal size significantly increases the quality of the results for both pointwise and contextual classification.

The automatic analysis of 3D point clouds has become a crucial task in photogrammetry, remote sensing and computer vision. Whereas modern range cameras simultaneously provide both range and intensity images with high frame rates, other devices can be used to obtain further information which could be quite valuable for tasks such as object detection or scene interpretation. In particular thermal information offers many advantages, since people can easily be detected as heat sources in typical indoor or outdoor environments and, furthermore, a variety of concealed objects such as heating pipes as well as structural properties such as defects in isolation may be observed. In this paper, we focus on thermal 3D mapping which allows to observe the evolution of a dynamic 3D scene over time. We present a fully automatic methodology consisting of four successive steps: (i) a radiometric correction, (ii) a geometric calibration, (iii) a robust approach for detecting reliable feature correspondences and (iv) a co-registration of 3D point cloud data and thermal information via a RANSAC-based EPnP scheme. For an indoor scene, we demonstrate that our methodology outperforms other recent approaches in terms of both accuracy and applicability. We additionally show that efficient straightforward techniques allow a categorization according to background, people, passive scene manipulation and active scene manipulation.

Obtaining accurate 3d descriptions in the thermal infrared (TIR) is a quite challenging task due to the low geometric resolutions of TIR cameras and the low number of strong features in TIR images. Combining the radiometric information of the thermal infrared with 3d data from another sensor is able to overcome most of the limitations in the 3d geometric accuracy. In case of dynamic scenes with moving objects or a moving sensor system, a combination with RGB cameras of Time-of-Flight (TOF) cameras is suitable. As a TOF camera is an active sensor in the near infrared (NIR) and the thermal infrared camera captures the radiation emitted by the objects in the observed scene, the combination of these two sensors for close range applications is independent from external illumination or textures in the scene. This article is focused on the fusion of data acquired both with a time-of-flight (TOF) camera and a thermal infrared (TIR) camera. As the radiometric behaviour of many objects differs between the near infrared used by the TOF camera and the thermal infrared spectrum, a direct co-registration with feature points in both intensity images leads to a high number of outliers. A fully automatic workflow of the geometric calibration of both cameras and the relative orientation of the camera system with one calibration pattern usable for both spectral bands is presented. Based on the relative orientation, a fusion of the TOF depth image and the TIR image is used for scene segmentation and people detection. An adaptive histogram based depth level segmentation of the 3d point cloud is combined with a thermal intensity based segmentation. The feasibility of the proposed method is demonstrated in an experimental setup with different geometric and radiometric influences that show the benefit of the combination of TOF intensity and depth images and thermal infrared images.

Due to ever more efficient and accurate laser scanning technologies, the analysis of 3D point clouds has become an important task in modern photogrammetry and remote sensing. To exploit the full potential of such data for structural analysis and object detection, reliable geometric features are of crucial importance. Since multiscale approaches have proved very successful for image-based applications, efforts are currently made to apply similar approaches on 3D point clouds. In this paper we analyse common geometric covariance features, pinpointing some severe limitations regarding their performance on varying scales. Instead, we propose a different feature type based on shape distributions known from object recognition. These novel features show a very reliable performance on a wide scale range and their results in classification outnumber covariance features in all tested cases.

3D scene analysis by automatically assigning 3D points a semantic label has become an issue of major interest in recent years. Whereas the tasks of feature extraction and classification have been in the focus of research, the idea of using only relevant and more distinctive features extracted from optimal 3D neighborhoods has only rarely been addressed in 3D lidar data processing. In this paper, we focus on the interleaved issue of extracting relevant, but not redundant features and increasing their distinctiveness by considering the respective optimal 3D neighborhood of each individual 3D point. We present a new, fully automatic and versatile framework consisting of four successive steps: (i) optimal neighborhood size selection, (ii) feature extraction, (iii) feature selection, and (iv) classification. In a detailed evaluation which involves 5 different neighborhood definitions, 21 features, 6 approaches for feature subset selection and 2 different classifiers, we demonstrate that optimal neighborhoods for individual 3D points significantly improve the results of scene interpretation and that the selection of adequate feature subsets may even further increase the quality of the derived results.

Mobile sensor devices offer great opportunities for automatic scene analysis and object recognition. Nowadays a new generation of ranging devices is available, like laser scanners which are small and light weighted. Concerning these improvements specific applications can be tackled. In this contribution we focus on vineyard monitoring for detecting and counting grape berries with a small, lightweight and low cost multi-echo laser scanner. Therefore a Hokuyo UTM-30LX-EW laser range finder is utilized for capturing the data in close range up to 1m. In order to process the data the following methodology is proposed: after smoothing and morphological techniques are applied on the laserscanning intensity and range images the number of visible grape berries is determined from the resulting segments. The approach performs with a detection accuracy of above 84%. The results reveal the high potential of such close range ranging devices for locating and counting grape berries. Thus, the methodology provides practical support for viticulture applications.

The automatic analysis of large 3D point clouds represents a crucial task in photogrammetry, remote sensing and computer vision. In this paper, we propose a new methodology for the semantic interpretation of such point clouds which involves feature relevance assessment in order to reduce both processing time and memory consumption. Given a standard benchmark dataset with 1.3 million 3D points, we first extract a set of 21 geometric 3D and 2D features. Subsequently, we apply a classifier-independent ranking procedure which involves a general relevance metric in order to derive compact and robust subsets of versatile features which are generally applicable for a large variety of subsequent tasks. This metric is based on 7 different feature selection strategies and thus addresses different intrinsic properties of the given data. For the example of semantically interpreting 3D point cloud data, we demonstrate the great potential of smaller subsets consisting of only the most relevant features with 4 different state-of-the-art classifiers. The results reveal that, instead of including as many features as possible in order to compensate for lack of knowledge, a crucial task such as scene interpretation can be carried out with only few versatile features and even improved accuracy.

In this paper, a new method for fusing optical and laserscanner data is presented for improved UAV-borne 3D mapping. We propose to equip an unmanned aerial vehicle (UAV) with a small platform which includes two sensors: a standard low-cost digital camera and a lightweight Hokuyo UTM-30LX-EW laserscanning device (210 g without cable). Initially, a calibration is carried out for the utilized devices. This involves a geometric camera calibration and the estimation of the position and orientation offset between the two sensors by lever-arm and bore-sight calibration. Subsequently, a feature tracking is performed through the image sequence by considering extracted interest points as well as the projected 3D laser points. These 2D results are fused with the measured laser distances and fed into a bundle adjustment in order to obtain a Simultaneous Localization and Mapping (SLAM). It is demonstrated that an improvement in terms of precision for the pose estimation is derived by fusing optical and laserscanner data.

The automatic and accurate alignment of multiple point clouds is a basic requirement for an adequate digitization, reconstruction and interpretation of large 3D environments. Due to the recent technological advancements, modern devices are available which allow for simultaneously capturing intensity and range images with high update rates. Hence, such devices can even be used for dynamic scene analysis and for rapid mapping which is particularly required for environmental applications and disaster management, but unfortunately, they also reveal severe restrictions. Facing challenges with respect to noisy range measurements, a limited non-ambiguous range, a limited field of view and the occurrence of scene dynamics, the adequate alignment of captured point clouds has to satisfy additional constraints compared to the classical registration of terrestrial laser scanning (TLS) point clouds for describing static scenes. In this paper, we propose a new methodology for point cloud registration which considers such constraints while maintaining the fundamental properties of high accuracy and low computational effort without relying on a good initial alignment or human interaction. Exploiting 2D image features and 2D/2D correspondences, sparse point clouds of physically almost identical 3D points are derived. Subsequently, these point clouds are aligned with a fast procedure directly taking into account the reliability of the detected correspondences with respect to geometric and radiometric information. The proposed methodology is evaluated and its performance is demonstrated for data captured with a moving sensor platform which has been designed for monitoring from low altitudes. Due to the provided reliability and a fast processing scheme, the proposed methodology offers a high potential for dynamic scene capture and analysis.

Unmanned Aerial Vehicles (UAVs) are nowadays promising platforms for capturing spatial information, because they are low cost solutions, which are easy to bring to the surveying field and can operate automatically. Usually these devices are equipped with visual sensors to support the navigation of the platform or to transmit observations of the environment to the operator. By collecting the data and processing the captured images even an estimation of the observed environment in form of 3D information is available. Therefore Simultaneous Localization and Mapping (SLAM) algorithms are well known for processing data which is captured in the visible domain. However, situations can occur where gathering visual information is difficult due to given limitations in form of low visibility. For example if soft obstacles in form of translucent materials are given in disaster scenarios with smoke and operating has still to be ensured, active optical sensors (e.g. laser scanners) are gaining interest because they can penetrate the soft obstacle and allow to acquire information behind it. A new lightweight (210 g), simplified and minimized scanning unit is now available which allows to capture multiple reflections for each transmitted laser pulse, namely the Hokuyo UTM-30LX-EW. With such a device, it is possible to overcome the above mentioned restrictions or limitations of low visibility by soft obstacles and even measure under critical circumstances. A multi-pulse system can provide accurate measurements on, within, and behind the soft obstacle. This research focuses on investigating the ability and performance of a laser scanner to penetrate the soft obstacle. Thus, investigations on a system that overcomes these limitations and provides a solution will be given. First promising experimental results considering soft obstacle are described.

Currently, enhanced types of active range imaging devices are available for capturing dynamic scenes. By using intensity and range images, data derived from different or the same range imaging devices can be fused. In this paper, an automatic image-based coregistration methodology is presented which uses a RANSAC-based scheme for the Efficient Perspective-n-Point (EPnP) algorithm. For evaluating the methodology, two different types of range imaging devices have been investigated, namely Microsoft Kinect and PMD [vision] CamCube 2.0. The data sets captured with the test devices have been compared to a reference device with respect to the absolute and relative accuracy. As the presented methodology can cope with different configurations concerning measurement principle, point density and range accuracy, it shows a high potential for automated data fusion for range imaging devices.

The estimation of the transformation parameters between different point clouds is still a crucial task as it is usually followed by scene reconstruction, object detection or object recognition. Therefore, the estimates should be as accurate as possible. Recent developments show that it is feasible to utilize both the measured range information and the reflectance information sampled as image, as 2D imagery provides additional information. In this paper, an image-based registration approach for TLS data is presented which consists of two major steps. In the first step, the order of the scans is calculated by checking the similarity of the respective reflectance images via the total number of SIFT correspondences between them. Subsequently, in the second step, for each SIFT correspondence the respective SIFT features are filtered with respect to their reliability concerning the range information and projected to 3D space. Combining the 3D points with 2D observations on a virtual plane yields 3D-to-2D correspondences from which the coarse transformation parameters can be estimated via a RANSAC-based registration scheme including the EPnP algorithm. After this coarse registration, the 3D points are again checked for consistency by using constraints based on the 3D distance, and, finally, the remaining 3D points are used for an ICP-based fine registration. Thus, the proposed methodology provides a fast, reliable, accurate and fully automatic image-based approach for the registration of unorganized point clouds without the need of a priori information about the order of the scans, the presence of regular surfaces or human interaction.

No abstract available.

Obtaining a 3D description of man-made and natural environments is a basic task in Computer Vision and Remote Sensing. To this end, laser scanning is currently one of the dominating techniques to gather reliable 3D information. The scanning principle inherently needs a certain time interval to acquire the 3D point cloud. On the other hand, new active sensors provide the possibility of capturing range information by images with a single measurement. With this new technique image-based active ranging is possible which allows capturing dynamic scenes, e.g. like walking pedestrians in a yard or moving vehicles. Unfortunately most of these range imaging sensors have strong technical limitations and are not yet sufficient for airborne data acquisition. It can be seen from the recent development of highly specialized (far-)range imaging sensors – so called flash-light lasers – that most of the limitations could be alleviated soon, so that future systems will be equipped with improved image size and potentially expanded operating range. The presented work is a first step towards the development of methods capable for application of range images in outdoor environments. To this end, an experimental setup was set up for investigating these proposed possibilities. With the experimental setup a measurement campaign was carried out and first results will be presented within this paper.

Obtaining a 3D description of man-made and natural environments is a basic task in Computer Vision, Photogrammetry and Remote Sensing. New active sensors provide the possibility of capturing range information by images with a single measurement. With this new technique, image-based active ranging is possible which allows for capturing dynamic scenes, e.g. with moving pedestrians or moving vehicles. The currently available range imaging devices usually operate within the close-infrared domain to capture range and furthermore active and passive intensity images. Depending on the application, a 3D description with additional spectral information such as thermal-infrared data can be helpful and offers new opportunities for the detection and interpretation of human subjects and interactions. Therefore, thermal-infrared data combined with range information is promising. In this paper, an approach for mapping thermal-infrared data on range data is proposed. First, a camera calibration is carried out for the range imaging system (PMD[vision] CamCube 2.0) and the thermal-infrared system (InfraTec VarioCAM hr). Subsequently, a registration of close-infrared and thermal infrared intensity images derived from different sensor devices is performed. In this context, wavelength independent properties are selected in order to derive point correspondences between the different spectral domains. Finally, the thermal infrared images are enhanced with information derived from data acquired with the range imaging device and the enhanced IR texture is projected onto the respective 3D point cloud data for gaining appropriate infrared-textured 3D models. The feasibility of the proposed methodology is demonstrated for an experimental setup which is well-suited for investigating these proposed possibilities. Hence, the presented work is a first step towards the development of methods for combined thermal-infrared and range representation.

Obtaining an appropriate 3D description of the local environment remains a challenging task in photogrammetric research. As terrestrial laser scanners (TLSs) perform a highly accurate, but time-dependent spatial scanning of the local environment, they are only suited for capturing static scenes. In contrast, new types of active sensors provide the possibility of simultaneously capturing range and intensity information by images with a single measurement, and the high frame rate also allows for capturing dynamic scenes. However, due to the limited field of view, one observation is not sufficient to obtain a full scene coverage and therefore, typically, multiple observations are collected from different locations. This can be achieved by either placing several fixed sensors at different known locations or by using a moving sensor. In the latter case, the relation between different observations has to be estimated by using information extracted from the captured data and then, a limited field of view may lead to problems if there are too many moving objects within it. Hence, a moving sensor platform with multiple and coupled sensor devices offers the advantages of an extended field of view which results in a stabilized pose estimation, an improved registration of the recorded point clouds and an improved reconstruction of the scene. In this paper, a new experimental setup for investigating the potentials of such multi-view range imaging systems is presented which consists of a moving cable car equipped with two synchronized range imaging devices. The presented setup allows for monitoring in low altitudes and it is suitable for getting dynamic observations which might arise from moving cars or from moving pedestrians. Relying on both 3D geometry and 2D imagery, a reliable and fully automatic approach for co-registration of captured point cloud data is presented which is essential for a high quality of all subsequent tasks. The approach involves using sparse point clouds as well as a new measure derived from the respective point quality. Additionally, an extension of this approach is presented for detecting special objects and, finally, decoupling sensor and object motion in order to improve the registration process. The results indicate that the proposed setup offers new possibilities for applications such as surveillance, scene reconstruction or scene interpretation.