Search

Working with pushbroom imagery in photogrammetry and remote sensing presents a fundamental challenge in object-to-image space transformation. For this transformation, accurate estimation of Exterior Orientation Parameters (EOPs) for each scanline is required. To tackle this challenge, Best Scanline Search or Determination (BSS/BSD) methods have been developed. However, the current BSS/BSD methods are not efficient for real-time applications due to their complex procedures and interpolations. This paper introduces a new non-iterative BSD method specifically designed for line-type pushbroom images. The method involves simulating a pair of sets of points, Simulated Control Points (SCOPs), and Simulated Check Points (SCPs), to train and test a Multilayer Perceptron (MLP) model. The model establishes a strong relationship between object and image spaces, enabling a direct transformation and determination of best scanlines. This proposed method does not rely on the Collinearity Equation (CE) or iterative search. After training, the MLP model is applied to the SCPs for accuracy assessment. The proposed method is tested on ten images with diverse landscapes captured by eight sensors, exploiting five million SCPs per image for statistical assessments. The Root Mean Square Error (RMSE) values range between 0.001 and 0.015 pixels across ten images, demonstrating the capability of achieving the desired sub-pixel accuracy within a few seconds. The proposed method is compared with conventional and state-of-the-art BSS/BSD methods, indicating its higher applicability regarding accuracy and computational efficiency. These results position the proposed BSD method as a practical solution for transforming object-to-image space, especially for real-time applications.

This study utilizes Thermal Infrared (TIR) imaging technology to detect hotspots in photovoltaic (PV) modules of solar power plants. Unmanned aerial vehicle (UAV)-based TIR imagery is crucial for efficiently analyzing fault detection in solar power plants. This research explores optimal operational parameters for generating high-quality TIR images using UAV technology. In addition to existing variables such as humidity, emissivity, height, wind speed, irradiance, and ambient temperature, newly considered variables including the angle of incidence between the target object and the thermal infrared camera are analyzed for their impact on TIR images. Based on the solar power plant’s tilt (20°) and the location coordinate data of the hotspot modules, the inner and outer products of the vectors were used to obtain the normal vector and angle of incidence of the solar power plant. It was discovered that the difference between measured TIR temperature data and Land Surface Temperature (LST) data varies with changes in the angle of incidence. The analysis presented in this study was conducted using multiple regression analysis to explore the relationships between dependent and independent variables. The Ordinary Least Squares (OLS) regression model employed was able to explain 63.6% of the variability in the dependent variable. Further, the use of the Condition Number (Cond. No.) and the Variance Inflation Factor (VIF) revealed that the multicollinearity among all variables was below 10, ensuring that the independence among variables was well-preserved while maintaining statistically significant correlations. Furthermore, a positive correlation was observed with the actual measured temperature values, while a negative correlation was observed between the TIR image data values and the angle of incidence. Moreover, it was found that an angle of incidence between 15° and 20° yields the closest similarity to LST temperature data. In conclusion, our research emphasizes the importance of adjusting the angle of incidence to 15–20° to enhance the accuracy of TIR imaging by mitigating overestimated TIR temperature values.

Rapid ripeness assessment of fruit after harvest is important to reduce post-harvest losses by sorting fruit according to the duration until they become ready to eat. However, there has been little research on non-destructive estimation of the ripeness and ripening speed of avocado fruit. Unlike previous methods, which classify the ripeness of fruit into a few categories (e.g., unripe and ripe) or indirectly estimate ripeness from its firmness, we developed a method using hyperspectral imaging coupled with deep learning regression to directly estimate the duration until ripeness of Hass avocado fruit. A set of 44,096 sub-images of 551 Hass avocado fruit images was used to train, validate, and test a convolutional neural network (CNN) to predict the number of days until ripeness. Training, validation, and test samples were generated as sub-images of Hass fruit images and were used to train a spectral–spatial residual network to estimate the duration to ripeness. We achieved predictions of duration to ripeness with an average error of 1.17 days per fruit on the test set. A series of experiments demonstrated that our deep learning regression approach outperformed classification approaches that rely on dimensionality reduction techniques such as principal component analysis. Our results show the potential for combining hyperspectral imaging with deep learning to estimate the ripeness stage of fruit, which could help to fine-tune avocado fruit sorting and processing.

Localization of pedestrians in 3D scene space from single RGB images is critical for various downstream applications. Current monocular approaches employ either the bounding box of pedestrians or the visible parts of their bodies for localization. Both approaches introduce additional error to the location estimation in the case of real-world scenarios – crowded environments with multiple occluded pedestrians. To overcome the limitation, this paper proposes a novel human pose-aware pedestrian localization framework to model poses of occluded pedestrians, where this enables accurate localization in image and ground space. This is done by proposing a light-weight neural network architecture, where this ensures a fast and accurate prediction of missing body parts for downstream applications. Comprehensive experiments on two real-world datasets demonstrate the effectiveness of the framework compared to state-of-the-art in predicting pedestrians missing body parts as well as pedestrian localization.

This paper reports the results of the ISPRS benchmark on indoor modelling. Reconstructed models submitted by 11 participating teams are evaluated on a dataset comprising 6 point clouds representing indoor environments of different complexity. The evaluation is based on measuring the completeness, correctness, and accuracy of the reconstructed wall elements through comparison with manually generated reference models. The results show that the performance of the methods varies across different datasets, but generally the reconstruction methods achieve better results for the point clouds with higher accuracy and density and fewer gaps, as well as the point clouds representing less complex environments. Filtering clutter points in a pre-processing step contributes to higher correctness, and making strong assumptions on the shape of the reconstructed elements contributes to higher completeness and accuracy for models of Manhattan World environments.

Pedestrian origin–destination (O–D) estimates that record traffic flows between origins and destinations, are essential for the management of pedestrian facilities including pedestrian flow simulation in the planning phase and crowd control in the operation phase. However, current O–D data collection techniques such as surveys, mobile sensing using GPS, Wi-Fi, and Bluetooth, and smart card data have the disadvantage that they are either time consuming and costly, or cannot provide complete O–D information for pedestrian facilities without entrances and exits or pedestrian flow inside the facilities. Due to the full coverage of CCTV cameras and the huge potential of image processing techniques, we address the challenges of pedestrian O–D estimation and propose an image-based O–D estimation framework. By identifying the same person in disjoint camera views, the O–D trajectory of each identity can be accurately generated. Then, state-of-the-art deep neural networks (DNNs) for person re-ID at different congestion levels were compared and improved. Finally, an O–D matrix based on trajectories was generated and the resident time was calculated, which provides recommendations for pedestrian facility improvement. The factors that affect the accuracy of the framework are discussed in this paper, which we believe could provide new insights and stimulate further research into the application of the Internet of cameras to intelligent transport infrastructure management.

Accurate and seamless vehicle positioning is fundamental for autonomous driving tasks in urban environments, requiring the provision of high-end measuring devices. Light Detection and Ranging (lidar) sensors, together with Global Navigation Satellite Systems (GNSS) receivers, are therefore commonly found onboard modern vehicles. In this paper, we propose an integration of lidar and GNSS code measurements at the observation level via a mixed measurement model. An Extended Kalman-Filter (EKF) is implemented to capture the dynamic of the vehicle movement, and thus, to incorporate the vehicle velocity parameters into the measurement model. The lidar positioning component is realized using point cloud registration through a deep neural network, which is aided by a high definition (HD) map comprising accurately georeferenced scans of the road environments. Experiments conducted in a densely built-up environment show that, by exploiting the abundant measurements of GNSS and high accuracy of lidar, the proposed vehicle positioning approach can maintain centimeter-to meter-level accuracy for the entirety of the driving duration in urban canyons.

Neste trabalho é apresentado um estudo que mostra a infl uência da precisão teórica das coordenadas 3D no processo de registro de dados RGB-D. Basicamente, três pontos principais são aqui discutidos: o registro dos pares de nuvens de pontos 3D adquiridas sucessivamente pelo sensor de imageamento; a ponderação das coordenadas 3D de pontos individuais; e a análise de consistência global. O registro dos pares de nuvens de pontos é feito em duas etapas. Na primeira etapa, os parâmetros de transformação são obtidos através de um conjunto de pontos, detectados pelo algoritmo SIFT, empregados no modelo de corpo rígido 3D, juntamente com o Método dos Mínimos Quadrados. A segunda etapa consiste em refi nar os parâmetros de transformação obtidos na etapa anterior empregando o algoritmo ICP baseado na abordagem ponto-a-ponto. A inclusão dos valores de pesos é feito baseado na propagação de covariâncias das equações de paralaxe. A análise de consistência global é empregada para refi nar o conjunto de parâmetros de translação simultaneamente. Experimentos em cenas com distância médias do sensor/alvo de 2 metros foram conduzidos para avaliar a efi ciência do método proposto e os resultados indicam a potencialidade do método com o valor médio das discrepâncias abaixo dos 2 centímetros.

Recently, deep convolutional neural networks (CNN) have become popular for indoor visual localisation, where the networks learn to regress the camera pose from images directly. However, these approaches perform a 3D image-based reconstruction of the indoor spaces beforehand to determine camera poses, which is a challenge for large indoor spaces. Synthetic images derived from 3D indoor models have been used to eliminate the requirement of 3D reconstruction. A limitation of the approach is the low accuracy that occurs as a result of estimating the pose of each image frame independently. In this article, a visual localisation approach is proposed that exploits the spatio-temporal information from synthetic image sequences to improve localisation accuracy. A deep Bayesian recurrent CNN is fine-tuned using synthetic image sequences obtained from a building information model (BIM) to regress the pose of real image sequences. The results of the experiments indicate that the proposed approach estimates a smoother trajectory with smaller inter-frame error as compared to existing methods. The achievable accuracy with the proposed approach is 1.6 m, which is an improvement of approximately thirty per cent compared to the existing approaches. A Keras implementation can be found in our Github repository.

Registration of point clouds is a central problem in many mapping and monitoring applications, such as outdoor and indoor mapping, high-speed railway track inspection, heritage documentation, building information modeling, and others. However, ensuring the global consistency of the registration is still a challenging task when there are multiple point clouds because the different scans should be transformed into a common coordinate frame. The aim of this paper is the registration of multiple terrestrial laser scanner point clouds. We present a plane-based matching algorithm to find plane-to-plane correspondences using a new parametrization based on complex numbers. The multiplication of complex numbers is based on analysis of the quadrants to avoid the ambiguity in the calculation of the rotation angle formed between normal vectors of adjacent planes. As a matching step may contain several matrix operations, our strategy is applied to reduce the number of mathematical operations. We also design a novel method for global refinement of terrestrial laser scanner data based on plane-to-plane correspondences. The rotation parameters are globally refined using operations of quaternion multiplication, while the translation parameters are refined using the parameters of planes. The global refinement is done non-iteratively. The experimental results show that the proposed plane-based matching algorithm efficiently finds plane correspondences in partial overlapping scans providing approximate values for the global registration, and indicate that an accuracy better than 8 cm can be achieved by using our global fine plane-to-plane registration method.

Automated reconstruction of Building Information Models (BIMs) from point clouds has been an intensive and challenging research topic for decades. Traditionally, 3D models of indoor environments are reconstructed purely by data-driven methods, which are susceptible to erroneous and incomplete data. Procedural-based methods such as the shape grammar are more robust to uncertainty and incompleteness of the data as they exploit the regularity and repetition of structural elements and architectural design principles in the reconstruction. Nevertheless, these methods are often limited to simple architectural styles: the so-called Manhattan design. In this paper, we propose a new method based on a combination of a shape grammar and a data-driven process for procedural modelling of indoor environments from a point cloud. The core idea behind the integration is to apply a stochastic process based on reversible jump Markov Chain Monte Carlo (rjMCMC) to guide the automated application of grammar rules in the derivation of a 3D indoor model. Experiments on synthetic and real data sets show the applicability of the method to efficiently generate 3D indoor models of both Manhattan and non-Manhattan environments with high accuracy, completeness, and correctness.

The recognition of locomotion activities (e.g., walking, running, still) is important for a wide range of applications like indoor positioning, navigation, location-based services, and health monitoring. Recently, there has been a growing interest in activity recognition using accelerometer data. However, when utilizing only acceleration-based features, it is difficult to differentiate varying vertical motion states from horizontal motion states especially when conducting user-independent classification. In this paper, we also make use of the newly emerging barometer built in modern smartphones, and propose a novel feature called pressure derivative from the barometer readings for user motion state recognition, which is proven to be effective for distinguishing vertical motion states and does not depend on specific users’ data. Seven types of motion states are defined and six commonly-used classifiers are compared. In addition, we utilize the motion state history and the characteristics of people’s motion to improve the classification accuracies of those classifiers. Experimental results show that by using the historical information and human’s motion characteristics, we can achieve user-independent motion state classification with an accuracy of up to 90.7%. In addition, we analyze the influence of the window size and smartphone pose on the accuracy.

In this paper, we describe the automatic extraction of centerlines of railroads. Mobile Laser Scanning systems are able to capture the 3D environment of the rail tracks with a high level of detail. Our approach first detects laser points that were reflected by the rail tracks, by making use of local properties such as parallelism and height in relation to neighboring objects. In the modeling stage, we present two approaches to determine the centerline location. The first approach generates center points in a data-driven manner by projecting rail track points to the parallel track, and taking the midpoint as initial center point. Next, a piecewise linear function is fitted through the center points to generate center points at a regular interval. The second approach models the rail track by fitting piecewise 3D track models to the rail track points. The model consists of a pair of two parallel rail tracks. The fitted pieces are smoothened by a Fourier series interpolation function. After that the centerline is implicitly determined by the geometric center of the pair of tracks. Reference data has been used to analyze the quality of our results, confirming that the position of the centerlines can be determined with an accuracy of 2–3 cm.

3D models of indoor environments are increasingly gaining importance due to the wide range of applications to which they can be subjected: from redesign and visualization to monitoring and simulation. These models usually exist only for newly constructed buildings; therefore, the development of automatic approaches for reconstructing 3D indoors from imagery and/or point clouds can make the process easier, faster and cheaper. Among the constructive elements defining a building interior, doors are very common elements and their detection can be very useful either for knowing the environment structure, to perform an efficient navigation or to plan appropriate evacuation routes. The fact that doors are topologically connected to walls by being coplanar, together with the unavoidable presence of clutter and occlusions indoors, increases the inherent complexity of the automation of the recognition process. In this work, we present a pipeline of techniques used for the reconstruction and interpretation of building interiors based on point clouds and images. The methodology analyses the visibility problem of indoor environments and goes in depth with door candidate detection. The presented approach is tested in real data sets showing its potential with a high door detection rate and applicability for robust and efficient envelope reconstruction.

Real-time traffic monitoring is essential in many novel applications, from traffic management to smart navigation systems. The large number of traffic cameras being integrated into urban infrastructures has enabled efficient traffic monitoring as an intervention in reducing traffic accidents and related casualties. In this paper, we focus on the problem of the automatic detection of anomalous driving behaviors, e.g., speeding or stopping on a bike lane, by using the traffic-camera feed that is available online. This can play an important role in personalized route-planning applications where, for instance, a user wants find the safest paths to get to a destination. We present an integrated system that accurately detects, tracks, and classifies vehicles using online traffic-camera feed.

With the rise of urban population, updated spatial information of indoor environments is needed in a growing number of applications. Navigational assistance for disabled or aged people, guidance for robots, augmented reality for gaming, and tourism or training emergency assistance units are just a few examples of the emerging applications requiring real three-dimensional (3D) spatial data of indoor scenes. This work proposes the use of point clouds for obstacle-aware indoor pathfinding. Point clouds are firstly used for reconstructing semantically rich 3D models of building structural elements in order to extract initial navigational information. Potential obstacles to navigation are classified in the point cloud and directly used to correct the path according to the mobility skills of different users. The methodology is tested in several real case studies for wheelchair and ordinary users. Experiments show that, after several iterations, paths are readapted to avoid obstacles.

Consumer-grade range cameras such as the Kinect sensor have the potential to be used in mapping applications where accuracy requirements are less strict. To realize this potential insight into the geometric quality of the data acquired by the sensor is essential. In this paper we discuss the calibration of the Kinect sensor, and provide an analysis of the accuracy and resolution of its depth data. Based on a mathematical model of depth measurement from disparity a theoretical error analysis is presented, which provides an insight into the factors influencing the accuracy of the data. Experimental results show that the random error of depth measurement increases with increasing distance to the sensor, and ranges from a few millimeters up to about 4 cm at the maximum range of the sensor. The quality of the data is also found to be influenced by the low resolution of the depth measurements.

The Dutch coast is characterized by sandy beaches flanked by dunes. Understanding the morphology of the coast is essential for defense against flooding of the hinterland. Because most dramatic changes of the beach and the first dune row happen during storms, it is important to assess the state of the coast immediately after a storm. This is expensive and difficult to organize with Airborne Laser Scanning (ALS). Therefore, the performance of a Land-based Mobile Mapping System (LMMS) in mapping a stretch of sandy Dutch coast of 6 km near the municipality of Egmond is evaluated in this research. A test data set was obtained by provider Geomaat using the StreetMapper LMMS system. Both the relative quality of laser point heights and of a derived Digital Terrain model (DTM) are assessed. First, the height precision of laser points is assessed a priori by random error propagation, and a posteriori by calculating the height differences between close-by points. In the a priori case, the result is a theoretical laser point precision of around 5 cm. In the a posteriori approach it is shown that on a flat beach a relative precision of 3 mm is achieved, and that almost no internal biases exist. In the second analysis, a DTM with a grid size of 1 m is obtained using moving least squares. Each grid point height includes a quality description, which incorporates both measurement precision and terrain roughness. Although some problems remain with the scanning height of 2 m, which causes shadow-effect behind low dunes, it is concluded that a laser LMMS enables the acquisition of a high quality DTM product, which is available within two days.

Ultra wideband (UWB) has been a popular technology for indoor positioning due to its high accuracy. However, in many indoor application scenarios UWB measurements are influenced by outliers under non-line of sight (NLOS) conditions. To detect and eliminate outlying UWB observations, we propose a UWB/Inertial Measurement Unit (UWB/IMU) fusion filter based on a Complementary Kalman Filter to track the errors of position, velocity and direction. By using the least squares method, the positioning residual of the UWB observation is calculated, the robustness factor of the observation is determined, and an observation weight is dynamically set. When the robustness factor does not exceed a pre-defined threshold, the observed value is considered trusted, and adaptive filtering is used to track the system state, while the abnormity of system state, which might be caused by IMU data exceptions or unreasonable noise settings, is detected by using Mahalanobis distance from the observation to the prior distribution. When the robustness factor exceeds the threshold, the observed value is considered abnormal, and robust filtering is used, whereby the impact of UWB data exceptions on the positioning results is reduced by exploiting Mahalanobis distance. Experimental results show that the observation error can be effectively estimated, and the proposed algorithm can achieve an improved positioning accuracy when affected by outlying system states of different quantity as well as outlying observations of different proportion.

AHN-2 is the second part of the Actueel Hoogtebestand Nederland project, which concerns the acquisition of high-resolution altimetry data over the entire Netherlands using airborne laser scanning. The accuracy assessment of laser altimetry data usually relies on comparing corresponding tie elements, often points or lines, in the overlapping strips. This paper proposes a new approach to strip adjustment and accuracy assessment of AHN-2 data by using planar features. In the proposed approach a transformation is estimated between two overlapping strips by minimizing the distances between points in one strip and their corresponding planes in the other. The planes and the corresponding points are extracted in an automated segmentation process. The point-to-plane distances are used as observables in an estimation model, whereby the parameters of a transformation between the two strips and their associated quality measures are estimated. We demonstrate the performance of the method for the accuracy assessment of the AHN-2 dataset over Zeeland province of The Netherlands. The results show vertical offsets of up to 4 cm between the overlapping strips, and horizontal offsets ranging from 2 cm to 34 cm.

This research investigates the use of image category classification to distinguish images posted to social media that are Witness Accounts of an event. Only images depicting observations of the event, captured by micro-bloggers at the event, are considered Witness Accounts. Identifying Witness Accounts from social media is important for services such as news, marketing and emergency response. Automated image category classification is essential due to the large number of images on social media and interest in identifying witnesses in near real time. This paper begins research of this emerging problem with an established procedure, using a bag-of-words method to create a vocabulary of visual words and classifier trained to categorize the encoded images. In order to test the procedure, a set of images were collected for case study events, Australian Football League matches, from Twitter. Evaluation shows an overall accuracy of 90% and precision and recall for both classes exceeding 83%.

Full-waveform lidar systems capture the complete backscattered signal from the interaction of the laser beam with targets located within the laser footprint. The resulting data have advantages over discrete return lidar, including higher accuracy of the range measurements and the possibility of retrieving additional returns from weak and overlapping pulses. In addition, radiometric characteristics of targets, e.g., target cross-section, can also be retrieved from the waveforms. However, waveform restoration and removal of the effect of the emitted system pulse from the returned waveform are critical for precise range measurement, 3D reconstruction and target cross-section extraction. In this paper, a sparsity-constrained regularization approach for deconvolution of the returned lidar waveform and restoration of the target cross-section is presented. Primal-dual interior point methods are exploited to solve the resulting nonlinear convex optimization problem. The optimal regularization parameter is determined based on the L-curve method, which provides high consistency in varied conditions. Quantitative evaluation and visual assessment of results show the superior performance of the proposed regularization approach in both removal of the effect of the system waveform and reconstruction of the target cross-section as compared to other prominent deconvolution approaches. This demonstrates the potential of the proposed approach for improving the accuracy of both range measurements and geophysical attribute retrieval. The feasibility and consistency of the presented approach in the processing of a variety of lidar data acquired under different system configurations is also highlighted.

With the growth of cities and increased urban population there is a growing demand for spatial information of large indoor environments.[...]