Search

Mobile lidar point clouds are commonly used for 3d mapping of road environments as they provide a rich, highly detailed geometric representation of objects on and around the road. However, raw lidar point clouds lack semantic information about the type of objects, which is necessary for various applications. Existing methods for the classification of objects in mobile lidar data, including state of the art deep learning methods, achieve relatively low accuracies, and a primary reason for this under-performance is the inadequacy of available 3d training samples to sufficiently train deep networks. In this paper, we propose a generative model for creating synthetic 3d point segments that can aid in improving the classification performance of mobile lidar point clouds. We train a 3d Adversarial Autoencoder (3dAAE) to generate synthetic point segments that exhibit a high resemblance to and share similar geometric features with real point segments. We evaluate the performance of a PointNet-like classifier trained with and without the synthetic point segments. The evaluation results support our hypothesis that training a classifier with training data augmented with synthetic samples leads to significant improvement in the classification performance. Specifically, our model achieves an F1 score of 0.94 for vehicles and pedestrians and 1.00 for traffic signs.

Individual tree extraction from urban mobile laser scanning (MLS) point clouds is important for many urban applications. Recently, deep learning-based semantic segmentation of urban MLS point clouds has achieved significant progress, which makes it possible to segment tree point clouds. However, tree segments often are spatially overlapping with varying shapes and incompleteness caused by occlusion, which makes individual tree extraction a challenging task. In this paper, we propose a novel top-down approach to extract individual trees from urban MLS point clouds. Firstly, a semantic segmentation deep network is applied to segment tree points from raw urban MLS point clouds, and then the segmented tree points are further grouped into a set of tree clusters using Euclidean distance clustering. Next, a pointwise direction embedding deep network (PDE-net) is proposed to predict the direction vectors pointing to tree centers for each tree cluster to enhance the boundaries of instance-level trees. After that, a direction aggregation-based strategy is developed to detect the tree centers for each tree cluster, and the clusters are classified into single-tree clusters and multi-tree clusters based on the number of detected tree centers. Finally, the single-tree clusters are directly extracted as individual trees, while the multi-tree clusters are further separated into instance-level trees based on our proposed accessible region growing algorithm combining the embedded pointwise directions and detected tree centers. Four MLS point clouds collected from different urban scenes were used to evaluate the performance of the proposed method. The precision, recall, and F-score of 0.96, 0.94, and 0.95, respectively, on these four datasets demonstrate the effectiveness of our approach. An implementation of the proposed method is available at: https://github.com/HiphonL/IndividualTreeExtraction .

Semantic segmentation is one of the fundamental tasks in understanding and applying urban scene point clouds. Recently, deep learning has been introduced to the field of point cloud processing. However, compared to images that are characterized by their regular data structure, a point cloud is a set of unordered points, which makes semantic segmentation a challenge. Consequently, the existing deep learning methods for semantic segmentation of point cloud achieve less success than those applied to images. In this article, we propose a novel method for urban scene point cloud semantic segmentation using deep learning. First, we use homogeneous supervoxels to reorganize raw point clouds to effectively reduce the computational complexity and improve the nonuniform distribution. Then, we use supervoxels as basic processing units, which can further expand receptive fields to obtain more descriptive contexts. Next, a sparse autoencoder (SAE) is presented for feature embedding representations of the supervoxels. Subsequently, we propose a regional relation feature reasoning module (RRFRM) inspired by relation reasoning network and design a multiscale regional relation feature segmentation network (MS-RRFSegNet) based on the RRFRM to semantically label supervoxels. Finally, the supervoxel-level inferences are transformed into point-level fine-grained predictions. The proposed framework is evaluated in two open benchmarks (Paris-Lille-3D and Semantic3D). The evaluation results show that the proposed method achieves competitive overall performance and outperforms other related approaches in several object categories. An implementation of our method is available at: https://github.com/HiphonL/MS_RRFSegNet .

Semantic segmentation is a fundamental task in understanding urban mobile laser scanning (MLS) point clouds. Recently, deep learning-based methods have become prominent for semantic segmentation of MLS point clouds, and many recent works have achieved state-of-the-art performance on open benchmarks. However, due to differences of objects across different scenes such as different height of buildings and different forms of the same road-side objects, the existing open benchmarks (namely source scenes) are often significantly different from the actual application datasets (namely target scenes). This results in underperformance of semantic segmentation networks trained using source scenes when applied to target scenes. In this paper, we propose a novel method to perform unsupervised scene adaptation for semantic segmentation of urban MLS point clouds. Firstly, we show the scene transfer phenomena in urban MLS point clouds. Then, we propose a new pointwise attentive transformation module (PW-ATM) to adaptively perform the data alignment. Next, a maximum classifier discrepancy-based (MCD-based) adversarial learning framework is adopted to further achieve feature alignment. Finally, an end-to-end alignment deep network architecture is designed for the unsupervised scene adaptation semantic segmentation of urban MLS point clouds. To experimentally evaluate the performance of our proposed approach, two large-scale labeled source scenes and two different target scenes were used for the training. Moreover, four actual application scenes are used for the testing. The experimental results indicated that our approach can effectively achieve scene adaptation for semantic segmentation of urban MLS point clouds.

Indoor localization is important for a variety of applications, such as location-based services, mobile social networks, and emergency response. Fusing spatial information is an effective way to achieve accurate indoor localization with little or with no need for extra hardware. However, the existing indoor localization methods that make use of spatial information are either computationally expensive or sensitive to the completeness of landmarks. In this article, we propose a novel, low-cost, high-accuracy indoor localization method based on a landmark graph. The experimental results show that the proposed method outperforms the state-of-the-art methods.

A major challenge in the application of state-of-the-art deep learning methods to the classification of mobile lidar data is the lack of sufficient training samples for different object categories. The transfer learning technique based on pre-trained networks, which is widely used in deep learning for image classification, is not directly applicable to point clouds, because pre-trained networks trained by a large number of samples from multiple sources are not available. To solve this problem, we design a framework incorporating a state-of-the-art deep learning network, i.e. VoxNet, and propose an extended Multiclass TrAdaBoost algorithm, which can be trained with complementary training samples from other source datasets to improve the classification accuracy in the target domain. In this framework, we first train the VoxNet model with the combined dataset and extract the feature vectors from the fully connected layer, and then use these to train the Multiclass TrAdaBoost. Experimental results show that the proposed method achieves both improvement in the overall accuracy and a more balanced performance in each category.

Multi-view video surveillance is a highly valuable tool to ensure the safety of the crowd in large public space. By utilizing complementary information captured by multiple cameras, the issue of li...

Localization of pedestrians in 3D scene space from single RGB images is critical for various downstream applications. Current monocular approaches employ either the bounding box of pedestrians or the visible parts of their bodies for localization. Both approaches introduce additional error to the location estimation in the case of real-world scenarios – crowded environments with multiple occluded pedestrians. To overcome the limitation, this paper proposes a novel human pose-aware pedestrian localization framework to model poses of occluded pedestrians, where this enables accurate localization in image and ground space. This is done by proposing a light-weight neural network architecture, where this ensures a fast and accurate prediction of missing body parts for downstream applications. Comprehensive experiments on two real-world datasets demonstrate the effectiveness of the framework compared to state-of-the-art in predicting pedestrians missing body parts as well as pedestrian localization.

In historical buildings, surface cracks are important indicators of potential structural damage. Natural disasters and indirect human factors, which are frequently encountered in recent periods, negatively affect historical buildings and structures. Fast and cost-effective crack detection methods play a key role in structural health monitoring of historical buildings. This paper presents methodologies for identifying concrete cracks using deep learning. We propose ReCRNet, a deep learning architecture designed for classifying images of cracks. The performance of the proposed method is evaluated and compared with state-of-the-art methods such as AlexNet, VGG 19, linear support vector machine (SVM), and decision tree (DT). The results show that ReCRNet achieves better performance in terms of accuracy, precision, recall, F1-score, and receiver operating characteristic (ROC), in comparison with the other crack classifiers. Accordingly, the proposed approach is recommended for automatic monitoring of historical buildings and building condition assessments.

Indoor localization is essential for healthcare, security, augmented reality gaming, and many other location-based services. There is currently a wealth of relevant literature on indoor localization. This article focuses on recent advances in indoor localization methods that use spatial context to improve the location estimation. Spatial context in the form of maps and spatial models have been used to improve the localization by constraining location estimates in the navigable parts of indoor environments. Landmarks such as doors and corners, which are also one form of spatial context, have proved useful in assisting indoor localization by correcting the localization error. This survey gives a comprehensive review of state-of-the-art indoor localization methods and localization improvement methods using maps, spatial models, and landmarks.

The classification of mobile Lidar data is challenged by the complexity of objects in the point clouds and the limited number of available training samples. Incomplete shape, noise points and uneven point density make the extraction of features from point clouds relatively arduous. Additionally, the difference in point density, and size and shape of objects, restricts the utilization of labelled samples from other sources. To solve this problem, we explore the possibility of improving the classification performance of a state-of-the-art deep learning method, Vox-Net, by using auxiliary training samples from a different dataset. We compare the performance of Vox-Net trained with and without the auxiliary dataset. The comparison shows that more instances can be recognized in classes with auxiliary data. At the same time, the performance in classes without complementary data can deteriorate due to the low number of samples in these categories. To achieve a balance in the performance for different categories, we further replace the classification layer of Vox-Net with AdaBoost. The AdaBoost classification displays good recognition ability in classes with few instances but decreases the overall accuracy.

Many current indoor localisation approaches need an initial location at the beginning of localisation. The existing visual approaches to indoor localisation perform a 3D reconstruction of the indoor spaces beforehand, for determining this initial location, which is challenging for large indoor spaces. In this research, we present a visual approach for indoor localisation that is eliminating the requirement of any image-based reconstruction of indoor spaces by using a 3D model. A deep Bayesian convolutional neural network is fine-tuned with synthetic images generated from a 3D model to estimate the camera pose of real images. The uncertainty of the estimated camera poses is modelled by sampling the outputs of the Bayesian network fine-tuned with synthetic images. The results of the experiments indicate that a localisation accuracy of 2 metres can be achieved using the proposed approach.

The ubiquity of cameras built in mobile devices has resulted in a renewed interest in image-based localisation in indoor environments where the global navigation satellite system (GNSS) signals are not available. Existing approaches for indoor localisation using images either require an initial location or need first to perform a 3D reconstruction of the whole environment using structure-from-motion (SfM) methods, which is challenging and time-consuming for large indoor spaces. In this paper, a visual localisation approach is proposed to eliminate the requirement of image-based reconstruction of the indoor environment by using a 3D indoor model. A deep convolutional neural network (DCNN) is fine-tuned using synthetic images obtained from the 3D indoor model to regress the camera pose. Results of the experiments indicate that the proposed approach can be used for indoor localisation in real-time with an accuracy of approximately 2 m.

This article presents an accurate and robust visual indoor localisation approach that not only is infrastructure-free, but also avoids accumulation error by taking advantage of (1) the widespread ubiquity of mobile devices with cameras and (2) the availability of 3D building models for most modern buildings. Localisation is performed by matching image sequences captured by a camera, with a 3D model of the building in a model-based visual tracking framework. Comprehensive evaluation of the approach with a photo-realistic synthetic dataset shows the robustness of the localisation approach under challenging conditions. Additionally, the approach is tested and evaluated on real data captured by a smartphone. The results of the experiments indicate that a localisation accuracy better than 10 cm can be achieved by using this approach. Since localisation errors do not accumulate the proposed approach is suitable for indoor localisation tasks for long periods of time and augmented reality applications, without requiring any local infrastructure. A MATLAB implementation can be found on https://github.com/debaditya-unimelb/BIM-Tracker .

Multi-camera pedestrians tracking is a challenging computer vision task. We propose a multi-camera tracker for monitoring pedestrians in an enclosed shopping environment. We assess the performance of the multi-camera tracker in a case study, tracking customers in a food and speciality market hall. Our multi-camera tracker tracks customers' walking between the stalls in the market. The information is useful for market management, visitor safety, and other potential application areas.

Recently, deep convolutional neural networks (CNN) have become popular for indoor visual localisation, where the networks learn to regress the camera pose from images directly. However, these approaches perform a 3D image-based reconstruction of the indoor spaces beforehand to determine camera poses, which is a challenge for large indoor spaces. Synthetic images derived from 3D indoor models have been used to eliminate the requirement of 3D reconstruction. A limitation of the approach is the low accuracy that occurs as a result of estimating the pose of each image frame independently. In this article, a visual localisation approach is proposed that exploits the spatio-temporal information from synthetic image sequences to improve localisation accuracy. A deep Bayesian recurrent CNN is fine-tuned using synthetic image sequences obtained from a building information model (BIM) to regress the pose of real image sequences. The results of the experiments indicate that the proposed approach estimates a smoother trajectory with smaller inter-frame error as compared to existing methods. The achievable accuracy with the proposed approach is 1.6 m, which is an improvement of approximately thirty per cent compared to the existing approaches. A Keras implementation can be found in our Github repository.

Indoor localization, navigation and mapping systems highly rely on the initial sensor pose information to achieve a high accuracy. Most existing indoor mapping and navigation systems cannot initialize the sensor poses automatically and consequently these systems cannot perform relocalization and recover from a pose estimation failure. For most indoor environments, a map or a 3D model is often available, and can provide useful information for relocalization. This paper presents a novel relocalization method for lidar sensors in indoor environments to estimate the initial lidar pose using a CNN pose regression network trained using a 3D model. A set of synthetic lidar frames are generated from the 3D model with known poses. Each lidar range image is a one-channel range image, used to train the CNN pose regression network from scratch to predict the initial sensor location and orientation. The CNN regression network trained by synthetic range images is used to estimate the poses of the lidar using real range images captured in the indoor environment. The results show that the proposed CNN regression network can learn from synthetic lidar data and estimate the pose of real lidar data with an accuracy of 1.9 m and 8.7 degrees.

In this paper, a novel approach to ego-motion estimation is proposed based on visual and inertial sensors, named Omnidirectional Visual-Inertial Odometry (OVIO). The proposed approach combines omnidirectional visual features with inertial measurements within the Multi-State Constraint Kalman Filter (MSCKF). In contrast with other visual inertial odometry methods that use visual features captured by perspective cameras, the proposed approach utilizes spherical images obtained by an omnidirectional camera to obtain more accurate estimates of the position and orientation of the camera. Because the standard perspective model is unsuitable for omnidirectional cameras, a measurement model on a plane tangent to the unit sphere rather than on the image plane is defined. The key hypothesis of OVIO is that a wider field of view allows the incorporation of more visual features from the surrounding environment, thereby improving the accuracy and robustness of the ego-motion estimation. Moreover, by using an omnidirectional camera, a situation where there is not enough texture is less likely to arise. Experimental evaluation of OVIO using synthetic and real video sequences captured by a fish-eye camera in both indoor and outdoor environments shows the superior performance of the proposed OVIO as compared to the MSCKF using a perspective camera in both positioning and attitude estimation.

Accurate and continuous positioning in global navigation satellite system (GNSS) deprived urban areas is crucial for autonomous vehicles and mobile mapping systems. To achieve this goal, we propose a stereo visual-inertial odometry approach using a multistate constraint Kalman filter (MSCKF). In contrast with the conventional MSCKF, in which an inertial navigation system (INS) propagates the vehicle motion, and then the propagation is corrected by measurements of salient features extracted from images of a single camera, we update the propagation with observations extracted from images of a stereo pair of cameras. This way, additional constraints across the stereo pairs of images are exploited to improve the pose estimation. Experimental results on several KITTI datasets show that the stereo MSCKF outperforms the mono one achieving an average positioning error of 0.9% of the trajectory length compared to 1.3% for the mono approach and 2.5% for INS-only integration. These results show that visual-inertial odometry has a promising potential for vehicle positioning in short periods of GNSS signal outage.

Locomotion activity recognition (LAR) is important for a number of applications, such as indoor localization, fitness tracking, and aged care. Existing methods usually use handcrafted features, which requires expert knowledge and is laborious, and the achieved result might still be suboptimal. To relieve the burden of designing and selecting features, we propose a deep learning method for LAR by using data from multiple sensors available on most smart devices. Experimental results show that the proposed method, which learns useful features automatically, outperforms conventional classifiers that require the hand-engineering of features. We also show that the combination of sensor data from four sensors (accelerometer, gyroscope, magnetometer, and barometer) achieves a higher accuracy than other combinations or individual sensors.

Robust and accurate step counting is important for indoor localization algorithms that rely on smartphone inertial sensors. Existing solutions for step counting do not consider users’ false walking state (e.g., when a user in still state uses her phone for texting, playing games, and watching movies), which results in the over-counting problem. In this paper, we propose a robust and accurate step counting algorithm to solve the overcounting problem caused by false walking. Experimental results show that the proposed algorithm outperforms the commonly-used peak detection-based method and can improve the step counting accuracy by 6.56% for normal walking case, 9.54% for free walking case, and by 58.92% for false walking case, which is a significant improvement in the accuracy and robustness. We also compare the proposed method with popular commercial step counting applications including S Health, i-Health, and Pedometer++, which shows that our method can achieve better accuracy.

The knowledge of the floor-level location of a user in a multi-storey building is important for many applications, especially for emergency response. Existing floor identification systems suffer from a variety of limitations such as low accuracy, the need for a time-consuming site survey, assumption of user encounters, knowledge of the initial floor, and/or poor applicability. In this paper, we propose a novel, zero-effort, deep learning-based floor identification system, called \textit{ZeeFi}. The proposed system uses the widely-available smartphone sensing to identify on which floor a user is located. By recognizing the ground floor automatically, the proposed system does not require site survey, initial floor knowledge, and other assumptions. To achieve accurate floor identification performance, we have developed a deep learning-based method. Experimental results show that the proposed system outperforms the state-of-the-art systems, and is very promising for large-scale deployment.

A fundamental building block towards efficient emergency management is the information on pedestrian route choices in emergency scenarios. Origin-destination (O-D) matrix is usually used to represent the information on the volume of pedestrians travelling from a set of origin zones to a set of destination zones. Conventional pedestrian O-D estimation methods, such as surveys, smart card, Bluetooth, Wi-Fi or CCTV cameras, require active user involvement or favorable lighting conditions. Motivated by the wide applications of RGB-D cameras that can provide an ambient illumination-independent channel, in this paper, we propose an RGB-D based pedestrian O-D estimation framework, which can work properly in fires, or low lighting conditions. Firstly, we train a joint convolutional neural network (CNN) on three modalities, including RGB, depth and infrared images. Subsequently, for evaluating person re-identification algorithm and O-D estimation framework, we collect and annotate a new top view RGB-D dataset considering different levels of service and lighting conditions, which we will make publicly available. Our experiment results demonstrate that the fusion of multiple modalities improves the pedestrian detection and reidentification accuracy by 13.4% compared with using single RGB modality. Especially in low lighting conditions where RGB cameras perform poorly, we can still achieve an accuracy of 72.8% using depth cameras.

In the recent years, indoor modelling and navigation has become a research of interest because many stakeholders require navigation assistance in various application scenarios. The navigational assistance for blind or wheelchair people, building crisis management such as fire protection, augmented reality for gaming, tourism or training emergency assistance units are just some of the direct applications of indoor modelling and navigation. Navigational information is traditionally extracted from 2D drawings or layouts. Real state of indoors, including opening position and geometry for both windows and doors, and the presence of obstacles is commonly ignored. In this work, a real indoor-path planning methodology based on 3D point clouds is developed. The value and originality of the approach consist on considering point clouds not only for reconstructing semantically-rich 3D indoor models, but also for detecting potential obstacles in the route planning and using these for readapting the routes according to the real state of the indoor depictured by the laser scanner.

Registration is often a prerequisite step in processing point clouds. While planar surfaces are suitable features for registration, most of the existing plane-based registration methods rely on iterative solutions for the estimation of transformation parameters from plane correspondences. This paper presents a new closed-form solution for the estimation of a rigid motion from a set of point–plane correspondences. The role of normalization is investigated and its importance for accurate plane fitting and plane-based registration is shown. The paper also presents a thorough evaluation of the closed-form solutions and compares their performance with the iterative solution in terms of accuracy, robustness, stability and efficiency. The results suggest that the closed-form solution based on point–plane correspondences should be the method of choice in point cloud registration as it is significantly faster than the iterative solution, and performs as well as or better than the iterative solution in most situations. The normalization of the point coordinates is also recommended as an essential preprocessing step for point cloud registration. An implementation of the closed-form solutions in MATLAB is available at: http://people.eng.unimelb.edu.au/kkhoshelham/research.html#directmotion .

In this letter, we present an adaptive coarse-to-fine registration method for 3-D indoor mapping using RGB-D data. We weight the 3-D points based on the theoretical random error of depth measurements and introduce a novel disparity-based model for an accurate and robust coarse-to-fine registration. Some feature extraction methods required by the method are also presented. First, our method exploits both visual and depth information to compute the initial transformation parameters. We employ scale-invariant feature transformation for extracting, detecting, and matching 2-D visual features, and their associated depth values are used to perform coarse registration. Then, we use an image-based segmentation technique for detecting regions in the RGB images. Their associated 3-D centroid and the correspondent disparity values are used to refine the initial transformation parameters. Finally, the loop-closure detection and a global adjustment of the complete sequence data are used to recognize when the camera has returned to a previously visited location and minimize the registration errors. The effectiveness of the proposed method is demonstrated with the Kinect data set. The experimental results show that the proposed method can properly map the indoor environment with a relative and absolute accuracy value of around 3-5 cm, respectively.

Rapid quality assessment of nuts is important to increase the shelf life and minimise the nut loss due to rancidity. Existing methods for nut quality estimation are usually slow and destructive. In this study, a quick and non-destructive method using hyperspectral imaging (HSI) coupled with deep learning classification was applied for the quality estimation of unblanched kernels in Canarium indicum categorized by peroxide values (PV). A set of 2300 sub-images of 289 C. indicum samples were used to train a convolutional neural network (CNN) to estimate quality levels. Series of ablation experiments showed that the highest overall accuracy of PV estimation on the test set reached 93.48%, with 95.59%, 90.00%, and 95.83% for good, medium, and poor quality nuts, respectively. The results indicate that deep learning classification of hyperspectral imagery offers a great potential for accurate, real-time, and non-destructive quality estimation of nuts in practical applications.

Dimensionality is one of the greatest challenges when deciphering hyperspectral imaging data. Although the multiband nature of the data is beneficial, algorithms are faced with a high computational load and statistical incompatibility due to the insufficient number of training samples. This is a hurdle to downstream applications. The combination of dimensionality and the real-world scenario of mixed pixels makes the identification and classification of imaging data challenging. Here, we address the complications of dimensionality using specific spectral indices from band combinations and spatial indices from texture measures for classification to better identify the classes. We classified spectral and combined spatial–spectral data and calculated measures of accuracy and entropy. A reduction in entropy and an overall accuracy of 80.50% was achieved when using the spectral–spatial input, compared with 65% for the spectral indices alone and 59.50% for the optimally determined principal components.

Registration of point clouds is a central problem in many mapping and monitoring applications, such as outdoor and indoor mapping, high-speed railway track inspection, heritage documentation, building information modeling, and others. However, ensuring the global consistency of the registration is still a challenging task when there are multiple point clouds because the different scans should be transformed into a common coordinate frame. The aim of this paper is the registration of multiple terrestrial laser scanner point clouds. We present a plane-based matching algorithm to find plane-to-plane correspondences using a new parametrization based on complex numbers. The multiplication of complex numbers is based on analysis of the quadrants to avoid the ambiguity in the calculation of the rotation angle formed between normal vectors of adjacent planes. As a matching step may contain several matrix operations, our strategy is applied to reduce the number of mathematical operations. We also design a novel method for global refinement of terrestrial laser scanner data based on plane-to-plane correspondences. The rotation parameters are globally refined using operations of quaternion multiplication, while the translation parameters are refined using the parameters of planes. The global refinement is done non-iteratively. The experimental results show that the proposed plane-based matching algorithm efficiently finds plane correspondences in partial overlapping scans providing approximate values for the global registration, and indicate that an accuracy better than 8 cm can be achieved by using our global fine plane-to-plane registration method.

Ultra wideband (UWB) has been a popular technology for indoor positioning due to its high accuracy. However, in many indoor application scenarios UWB measurements are influenced by outliers under non-line of sight (NLOS) conditions. To detect and eliminate outlying UWB observations, we propose a UWB/Inertial Measurement Unit (UWB/IMU) fusion filter based on a Complementary Kalman Filter to track the errors of position, velocity and direction. By using the least squares method, the positioning residual of the UWB observation is calculated, the robustness factor of the observation is determined, and an observation weight is dynamically set. When the robustness factor does not exceed a pre-defined threshold, the observed value is considered trusted, and adaptive filtering is used to track the system state, while the abnormity of system state, which might be caused by IMU data exceptions or unreasonable noise settings, is detected by using Mahalanobis distance from the observation to the prior distribution. When the robustness factor exceeds the threshold, the observed value is considered abnormal, and robust filtering is used, whereby the impact of UWB data exceptions on the positioning results is reduced by exploiting Mahalanobis distance. Experimental results show that the observation error can be effectively estimated, and the proposed algorithm can achieve an improved positioning accuracy when affected by outlying system states of different quantity as well as outlying observations of different proportion.

Currently, Digital Elevation Models (DEMs) are the main source for representing Earth’s surface and have been an integral part of any geospatial analysis, in particular, geomorphometric research. Among existing techniques, photogrammetry is considered as a common method for obtaining high-resolution elevation data, especially over large or/and inaccessible areas. However, this DEMs are disposed to a severe amount of uncertainty. The aim of this research is to provide an optimal filtering strategy that can remove possible errors and improve the quality of raw photogrammetric DEMs for geomorphometric research over regions with relatively low relief. Results reveal that a combination of three digital filters, namely Gaussian filter, Median filter, and Slope Based filter, is able to reduce different sources of errors and improve the elevation accuracy of DEM by more than 5%, resulting in improved quality of the derived geomorphometric parameters.

The precision of Terrestrial Laser Scanning (TLS) data depends mainly on the inherent random range error, which hinders extraction of small details from TLS measurements. New post processing algorithms have been developed that reduce or eliminate the noise and therefore enable modelling details at a smaller scale than one would traditionally expect. The aim of this research is to find the optimum denoising method such that the corrected TLS data provides a reliable estimation of small-scale rock joint roughness. Two wavelet-based denoising methods are considered, namely Discrete Wavelet Transform (DWT) and Stationary Wavelet Transform (SWT), in combination with different thresholding procedures. The question is, which technique provides a more accurate roughness estimates considering (i) wavelet transform (SWT or DWT), (ii) thresholding method (fixed-form or penalised low) and (iii) thresholding mode (soft or hard). The performance of denoising methods is tested by two analyses, namely method noise and method sensitivity to noise. The reference data are precise Advanced TOpometric Sensor (ATOS) measurements obtained on 20 × 30 cm rock joint sample, which are for the second analysis corrupted by different levels of noise. With such a controlled noise level experiments it is possible to evaluate the methods’ performance for different amounts of noise, which might be present in TLS data. Qualitative visual checks of denoised surfaces and quantitative parameters such as grid height and roughness are considered in a comparative analysis of denoising methods. Results indicate that the preferred method for realistic roughness estimation is DWT with penalised low hard thresholding.

Indoor positioning is a fundamental requirement of many indoor location-based services and applications. In this paper, we explore the potential of low-cost and widely available visual and inertial sensors for indoor positioning. We describe the Visual-Inertial Odometry (VIO) approach and propose a measurement model for omnidirectional visual-inertial odometry (OVIO). The results of experiments in two simulated indoor environments show that the OVIO approach outperforms VIO and achieves a positioning accuracy of 1.1 % of the trajectory length.

Nowadays, lidar is widely used in cultural heritage documentation, urban modeling, and driverless car technology for its fast and accurate 3D scanning ability. However, full exploitation of the potential of point cloud data for efficient and automatic object recognition remains elusive. Recently, feature-based methods have become very popular in object recognition on account of their good performance in capturing object details. Compared with global features describing the whole shape of the object, local features recording the fractional details are more discriminative and are applicable for object classes with considerable similarity. In this paper, we propose a two-step classification approach based on point feature histograms and the bag-of-features method for automatic recognition of similar objects in mobile lidar point clouds. Lamp post, street light and traffic sign are grouped as one category in the first-step classification for their inter similarity compared with tree and vehicle. A finer classification of the lamp post, street light and traffic sign based on the result of the first-step classification is implemented in the second step. The proposed two-step classification approach is shown to yield a considerable improvement over the conventional one-step classification approach.

Existing 3D indoor mapping of RGB-D data are prominently point-based and feature-based methods. In most cases iterative closest point (ICP) and its variants are generally used for pairwise registration process. Considering that the ICP algorithm requires an relatively accurate initial transformation and high overlap a weighted closed-form solution for RGB-D data registration is proposed. In this solution, we weighted and normalized the 3D points based on the theoretical random errors and the dual-number quaternions are used to represent the 3D rigid body motion. Basically, dual-number quaternions provide a closed-form solution by minimizing a cost function. The most important advantage of the closed-form solution is that it provides the optimal transformation in one-step, it does not need to calculate good initial estimates and expressively decreases the demand for computer resources in contrast to the iterative method. Basically, first our method exploits RGB information. We employed a scale invariant feature transformation (SIFT) for extracting, detecting, and matching features. It is able to detect and describe local features that are invariant to scaling and rotation. To detect and filter outliers, we used random sample consensus (RANSAC) algorithm, jointly with an statistical dispersion called interquartile range (IQR). After, a new RGB-D loop-closure solution is implemented based on the volumetric information between pair of point clouds and the dispersion of the random errors. The loop-closure consists to recognize when the sensor revisits some region. Finally, a globally consistent map is created to minimize the registration errors via a graph-based optimization. The effectiveness of the proposed method is demonstrated with a Kinect dataset. The experimental results show that the proposed method can properly map the indoor environment with an absolute accuracy around 1.5% of the travel of a trajectory.

Indoor localization is important for a variety of applications such as location-based services, mobile social networks, and emergency response. Fusing spatial information is an effective way to achieve accurate indoor localization with little or with no need for extra hardware. However, existing indoor localization methods that make use of spatial information are either too computationally expensive or too sensitive to the completeness of landmark detection. In this paper, we solve this problem by using the proposed landmark graph. The landmark graph is a directed graph where nodes are landmarks (e.g., doors, staircases, and turns) and edges are accessible paths with heading information. We compared the proposed method with two common Dead Reckoning (DR)-based methods (namely, Compass + Accelerometer + Landmarks and Gyroscope + Accelerometer + Landmarks) by a series of experiments. Experimental results show that the proposed method can achieve 73% accuracy with a positioning error less than 2.5 meters, which outperforms the other two DR-based methods.

The fusion of ultra-wideband (UWB) and inertial measurement unit (IMU) is an effective solution to overcome the challenges of UWB in nonline-of-sight (NLOS) conditions and error accumulation of inertial positioning in indoor environments. However, existing systems are based on foot-mounted or body-worn IMUs, which limit the application of the system to specific practical scenarios. In this paper, we propose the fusion of UWB and pedestrian dead reckoning (PDR) using smartphone IMU, which has the potential to provide a universal solution to indoor positioning. The PDR algorithm is based on low-pass filtering of acceleration data and time thresholding to estimate the step length. According to different movement patterns of pedestrians, such as walking and running, several step models are comparatively analyzed to determine the appropriate model and related parameters of the step length. For the PDR direction calculation, the Madgwick algorithm is adopted to improve the calculation accuracy of the heading algorithm. The proposed UWB/PDR fusion algorithm is based on the extended Kalman filter (EKF), in which the Mahalanobis distance from the observation to the prior distribution is used to suppress the influence of abnormal UWB data on the positioning results. Experimental results show that the algorithm is robust to the intermittent noise, continuous noise, signal interruption, and other abnormalities of the UWB data.

This paper describes a method for automated detection of temporary cars in Mobile LiDAR point clouds. It consists of a segment-based classification of static cars and a comparison of data from two sensors to identify moving cars. Two segmentation methods are used to extract the ground and group the above-ground points into objects. From each segmented object a number of features are extracted, and a classification strengthened by feature selection is performed to classify temporary cars. We evaluate the performance of two different classifiers trained with a training set including 117 temporary cars, and show classification accuracies of up to 92%. We also investigate a method for identifying moving cars based on the distance between corresponding segments in the point clouds captured by the two scanning sensors, and report an overall accuracy of 61%.

We propose and analyse methods to efficiently register point clouds based on plane correspondences. This is relevant in man-made environments, where most objects are bounded by planar surfaces. Based on a segmentation of the point clouds into planar regions and matches of planes in different point clouds, we (1) optimally estimate the relative pose(s); (2) provide three direct solutions, of which two take the uncertainty of the given planes into account; and (3) analyse the loss in accuracy of the direct solutions as compared to the optimal solution. The paper presents the different solutions, derives their uncertainty especially of the suboptimal direct solutions, and compares their accuracy based on simulated and real data. We show that the direct methods that exploit the uncertainty of the planes lead to a maximum loss of 2.76 in accuracy of the estimated motion parameters in terms of the achieved standard deviations compared to the optimal estimates. We also show that the results are more accurate than the classical iterative closest point and iterative closest plane method, but the estimation procedures have a significantly lower computational complexity. We finally show how to generalize the estimation scheme to simultaneously register multiple point clouds.1

Human activity recognition is important for a large number of applications including indoor localization. Existing methods usually involve manually-designed features, which require expert knowledge and are laborious. Also, previous works use only the accelerometer for activity recognition, which may fail to recognize some complex activities. In this paper, we propose a deep learning-based method for locomotion activity recognition by using the combination of data from multiple smartphone built-in sensors. Eight types of locomotion activities are identified including the new ‘False Motion’ activity introduced for the first time in this work. Experimental results show that the proposed method, which learns useful features automatically, outperforms conventional classifiers that require hand-engineering of features. Also, using data from multiple sensors helps to improve recognition accuracy by about 10% compared to that using accelerometer data only.

We present an Omnidirectional Visual-Inertial Odometry (OVIO) approach based on Multi-State Constraint Kalman Filtering (MSCKF) to estimate the ego-motion of a moving platform. Instead of considering visual measurements on image plane, we use individual planes for each point that are tangent to the unit sphere and normal to the corresponding measurement ray. This way, we combine spherical images captured by omnidirectional camera with inertial measurements within the filtering method MSCKF. The key hypothesis of OVIO is that a wider field of view allows incorporating more visual features from the surrounding environment, thereby improving the accuracy and robustness of the motion estimation. Moreover, by using an omnidirectional camera, it is less likely to end up in a situation where there is not enough texture. We provide an evaluation of OVIO using synthetic and real video sequences captured by a fish-eye camera, and compare the performance with MSCKF using a perspective camera. The results show the superior performance of the proposed OVIO.

This research investigates the use of image category classification to distinguish images posted to social media that are Witness Accounts of an event. Only images depicting observations of the event, captured by micro-bloggers at the event, are considered Witness Accounts. Identifying Witness Accounts from social media is important for services such as news, marketing and emergency response. Automated image category classification is essential due to the large number of images on social media and interest in identifying witnesses in near real time. This paper begins research of this emerging problem with an established procedure, using a bag-of-words method to create a vocabulary of visual words and classifier trained to categorize the encoded images. In order to test the procedure, a set of images were collected for case study events, Australian Football League matches, from Twitter. Evaluation shows an overall accuracy of 90% and precision and recall for both classes exceeding 83%.

Accurate positioning of moving vehicles in GNSS-deprived urban areas is important for autonomous vehicles and mobile mapping systems. In this paper, we discuss various approaches to vehicle positioning in the absence of GNSS signals. We explore the potential of visual-inertial odometry, and present experimental results that show visual-inertial odometry can provide positioning accuracies up to 0.9% of the trajectory length, which is promising for vehicle positioning in short periods of GNSS signal outage.

A global registration is often insufficient for estimating dendrometric characteristics of trees because individual branches of the same tree may exhibit different positions between two scanning procedures. Therefore, we introduce a localized approach to register point clouds of botanic trees. Given two roughly registered point clouds PC1 and PC2 of a tree, we apply a skeletonization method to both point clouds. Based on these two skeletons, initial correspondences between branch segments of both point clouds are established to estimate local transformation parameters. The transformation estimation relies on minimizing the distance between the points in PC1 and the skeleton of PC2. The performance of the method is demonstrated on two example trees. It is shown that significant improvements can be achieved for the registration of fine branches. These improvements are quantified as the residual point-to-line distances before and after the localized fine registration. In our experiment, the residual error after the local registration is on an average of 5 mm over 90 skeleton segments, which is about three times smaller than the average residual error of the initial rough registration.

Identifying damaged buildings after natural disasters such as earthquake is important for the planning of recovery actions. We present a segment-based approach to classifying damaged building roofs in aerial laser scanning data. A challenge in the supervised classification of point segments is the generation of training samples, which is difficult because of the complexity of interpreting point clouds. We evaluate the performance of three different classifiers trained with a small set of training samples and show that feature selection improves the training and the accuracy of the resulting classification. When trained with 50 training samples, a linear discriminant classifier using a subset of six features reaches a classification accuracy of 85%.

Indoor localization is important for a variety of applications such as emergency response, shopping guide, and location-based services. Localization based on smartphone inertial sensors is one of the most widely-used indoor localization techniques since it can provide continuous and real-time locations without requiring additional infrastructure. However, it suffers from the accumulated error problem, which can be addressed by using sensory landmarks. In this paper, we first introduce the concept of sensory landmarks, and then show how different types of sensory landmarks can be detected. The method for estimating the locations of sensory landmarks is also given.

Hyperspectral remote sensing's ability to capture spectral information of targets in very narrow bandwidths gives rise to many intrinsic applications. However, the major limiting disadvantage to its applicability is its dimensionality, known as the Hughes Phenomenon. Traditional classification and image processing approaches fail to process data along many contiguous bands due to inadequate training samples. Another challenge of successful classification is to deal with the real world scenario of mixed pixels i.e. presence of more than one class within a single pixel. An attempt has been made to deal with the problems of dimensionality and mixed pixels, with an objective to improve the accuracy of class identification. In this paper, we discuss the application of indices to cope with the disadvantage of the dimensionality of the Airborne Prism EXperiment (APEX) hyperspectral Open Science Dataset (OSD) and to improve the classification accuracy using the Possibilistic c–Means (PCM) algorithm. This was used for the formulation of spectral and spatial indices to describe the information in the dataset in a lesser dimensionality. This reduced dimensionality is used for classification, attempting to improve the accuracy of determination of specific classes. Spectral indices are compiled from the spectral signatures of the target and spatial indices have been defined using texture analysis over defined neighbourhoods. The classification of 20 classes of varying spatial distributions was considered in order to evaluate the applicability of spectral and spatial indices in the extraction of specific class information. The classification of the dataset was performed in two stages; spectral and a combination of spectral and spatial indices individually as input for the PCM classifier. In addition to the reduction of entropy, while considering a spectral-spatial indices approach, an overall classification accuracy of 80.50% was achieved, against 65% (spectral indices only) and 59.50% (optimally determined principal components).