Search

Predicting the ripening time of avocado fruit accurately could improve fruit storage and decrease food waste. No reasonable method exists for predicting the postharvest ripening time of avocado fruit during transport, storage or retail display. Here, hyperspectral imaging ranging from 388 to 1005 nm with 462 bands was applied to 316 ‘Hass’ and 160 ‘Shepard’ mature, unripe avocado fruit to predict how many days it took for individual fruit to become ripe. Three models were developed using partial least squares regression (PLSR), deep convolutional neural network (DCNN) regression and DCNN classification. Our PLSR models provided coefficients of determination (R2) of 0.76 and 0.50 and root mean squared errors (RMSE) of 1.20 and 1.13 days for ‘Hass’ and ‘Shepard’ fruit, respectively. The DCNN-based regression models produced similar results with R2 of 0.77 and 0.59, and RMSEs of 1.43 and 0.94 days for ‘Hass’ and ‘Shepard’ fruit, respectively. The prediction accuracies and RMSEs from DCNN classification models, respectively, were 67.28% and 1.52 days for ‘Hass’ and 64.06% and 1.03 days for ‘Shepard’. Our study demonstrates that the spectral reflectance of the skin of mature, unripe ‘Hass’ and ‘Shepard’ fruit provides adequate information to predict ripening time and, thus, has the potential to improve postharvest processing and reduce postharvest losses of avocado fruit.

Mobile lidar point clouds are commonly used for 3d mapping of road environments as they provide a rich, highly detailed geometric representation of objects on and around the road. However, raw lidar point clouds lack semantic information about the type of objects, which is necessary for various applications. Existing methods for the classification of objects in mobile lidar data, including state of the art deep learning methods, achieve relatively low accuracies, and a primary reason for this under-performance is the inadequacy of available 3d training samples to sufficiently train deep networks. In this paper, we propose a generative model for creating synthetic 3d point segments that can aid in improving the classification performance of mobile lidar point clouds. We train a 3d Adversarial Autoencoder (3dAAE) to generate synthetic point segments that exhibit a high resemblance to and share similar geometric features with real point segments. We evaluate the performance of a PointNet-like classifier trained with and without the synthetic point segments. The evaluation results support our hypothesis that training a classifier with training data augmented with synthetic samples leads to significant improvement in the classification performance. Specifically, our model achieves an F1 score of 0.94 for vehicles and pedestrians and 1.00 for traffic signs.

Individual tree extraction from urban mobile laser scanning (MLS) point clouds is important for many urban applications. Recently, deep learning-based semantic segmentation of urban MLS point clouds has achieved significant progress, which makes it possible to segment tree point clouds. However, tree segments often are spatially overlapping with varying shapes and incompleteness caused by occlusion, which makes individual tree extraction a challenging task. In this paper, we propose a novel top-down approach to extract individual trees from urban MLS point clouds. Firstly, a semantic segmentation deep network is applied to segment tree points from raw urban MLS point clouds, and then the segmented tree points are further grouped into a set of tree clusters using Euclidean distance clustering. Next, a pointwise direction embedding deep network (PDE-net) is proposed to predict the direction vectors pointing to tree centers for each tree cluster to enhance the boundaries of instance-level trees. After that, a direction aggregation-based strategy is developed to detect the tree centers for each tree cluster, and the clusters are classified into single-tree clusters and multi-tree clusters based on the number of detected tree centers. Finally, the single-tree clusters are directly extracted as individual trees, while the multi-tree clusters are further separated into instance-level trees based on our proposed accessible region growing algorithm combining the embedded pointwise directions and detected tree centers. Four MLS point clouds collected from different urban scenes were used to evaluate the performance of the proposed method. The precision, recall, and F-score of 0.96, 0.94, and 0.95, respectively, on these four datasets demonstrate the effectiveness of our approach. An implementation of the proposed method is available at: https://github.com/HiphonL/IndividualTreeExtraction .

Semantic segmentation is one of the fundamental tasks in understanding and applying urban scene point clouds. Recently, deep learning has been introduced to the field of point cloud processing. However, compared to images that are characterized by their regular data structure, a point cloud is a set of unordered points, which makes semantic segmentation a challenge. Consequently, the existing deep learning methods for semantic segmentation of point cloud achieve less success than those applied to images. In this article, we propose a novel method for urban scene point cloud semantic segmentation using deep learning. First, we use homogeneous supervoxels to reorganize raw point clouds to effectively reduce the computational complexity and improve the nonuniform distribution. Then, we use supervoxels as basic processing units, which can further expand receptive fields to obtain more descriptive contexts. Next, a sparse autoencoder (SAE) is presented for feature embedding representations of the supervoxels. Subsequently, we propose a regional relation feature reasoning module (RRFRM) inspired by relation reasoning network and design a multiscale regional relation feature segmentation network (MS-RRFSegNet) based on the RRFRM to semantically label supervoxels. Finally, the supervoxel-level inferences are transformed into point-level fine-grained predictions. The proposed framework is evaluated in two open benchmarks (Paris-Lille-3D and Semantic3D). The evaluation results show that the proposed method achieves competitive overall performance and outperforms other related approaches in several object categories. An implementation of our method is available at: https://github.com/HiphonL/MS_RRFSegNet .

Semantic segmentation is a fundamental task in understanding urban mobile laser scanning (MLS) point clouds. Recently, deep learning-based methods have become prominent for semantic segmentation of MLS point clouds, and many recent works have achieved state-of-the-art performance on open benchmarks. However, due to differences of objects across different scenes such as different height of buildings and different forms of the same road-side objects, the existing open benchmarks (namely source scenes) are often significantly different from the actual application datasets (namely target scenes). This results in underperformance of semantic segmentation networks trained using source scenes when applied to target scenes. In this paper, we propose a novel method to perform unsupervised scene adaptation for semantic segmentation of urban MLS point clouds. Firstly, we show the scene transfer phenomena in urban MLS point clouds. Then, we propose a new pointwise attentive transformation module (PW-ATM) to adaptively perform the data alignment. Next, a maximum classifier discrepancy-based (MCD-based) adversarial learning framework is adopted to further achieve feature alignment. Finally, an end-to-end alignment deep network architecture is designed for the unsupervised scene adaptation semantic segmentation of urban MLS point clouds. To experimentally evaluate the performance of our proposed approach, two large-scale labeled source scenes and two different target scenes were used for the training. Moreover, four actual application scenes are used for the testing. The experimental results indicated that our approach can effectively achieve scene adaptation for semantic segmentation of urban MLS point clouds.

Indoor localization is important for a variety of applications, such as location-based services, mobile social networks, and emergency response. Fusing spatial information is an effective way to achieve accurate indoor localization with little or with no need for extra hardware. However, the existing indoor localization methods that make use of spatial information are either computationally expensive or sensitive to the completeness of landmarks. In this article, we propose a novel, low-cost, high-accuracy indoor localization method based on a landmark graph. The experimental results show that the proposed method outperforms the state-of-the-art methods.

Knowledge of deteriorations within tree trunks is critical for arborists to conduct individual tree health assessments. Sonic tree tomography, a non-destructive technique using sound waves, has been widely used to estimate the size and shape of internal decay based on sound wave velocity variations. However, it has commonly been applied to 2D horizontal or vertical cross sections and its accuracy is questionable due to the poor approximation of the shape of the cross section. This paper proposes an integration of close-range photogrammetry and sonic tomography to enable accurate reconstruction of the exterior and interior of the tree trunk in 3D. The internal wood quality is represented by the spatially interpolated sound wave velocities, using the time of flight of the sound waves and the coordinates of the acoustic sensors obtained from the photogrammetric model. Experimental results show that the proposed approach provides a realistic 3D visualisation of the size, shape and location of the internal deteriorations.

A major challenge in the application of state-of-the-art deep learning methods to the classification of mobile lidar data is the lack of sufficient training samples for different object categories. The transfer learning technique based on pre-trained networks, which is widely used in deep learning for image classification, is not directly applicable to point clouds, because pre-trained networks trained by a large number of samples from multiple sources are not available. To solve this problem, we design a framework incorporating a state-of-the-art deep learning network, i.e. VoxNet, and propose an extended Multiclass TrAdaBoost algorithm, which can be trained with complementary training samples from other source datasets to improve the classification accuracy in the target domain. In this framework, we first train the VoxNet model with the combined dataset and extract the feature vectors from the fully connected layer, and then use these to train the Multiclass TrAdaBoost. Experimental results show that the proposed method achieves both improvement in the overall accuracy and a more balanced performance in each category.

Meat shrinks and assumes an irregular shape during heating due to the varying distribution of connective tissue and extracellular spaces. The terrestrial 3D laser scanning technology is proposed as an alternative method to manual measurements to estimate the volume of irregularly shaped meat cuboids and to predict the cooking loss based on the heating induced volume shrinkage. Cuboids from aged pork loins (longissimus lumborum, n = 12) were heated at 50, 60, 70 or 80 °C for 30 min. Two methods of 3D reconstruction and volume estimation of the pork cuboids by laser scanning were used; without a base scan (laser-B) and with inclusion of a base scan (laser+B), as well as two methods based on manual caliper measurements of all twelve edges (caliper-12) or of three edges in each direction (caliper-3). Both laser scanning methods (Laser+B and Laser-B) resulted in greater volume estimates for the raw samples than the caliper-12 and caliper-3 measurements (38.3, 39.4 compared to 33.9, 34.8 cm3, respectively). Cooking loss across the different temperatures could be best predicted by the caliper-based perimeter shrinkage (r = 0.94, P 0.05) with the cooking loss. 3D laser scanning technologies can be considered by the food industry for 3D reconstruction and volume estimation, however improvements are needed in the data processing to allow for better predictability of meat quality attributes.

This paper reports the results of the ISPRS benchmark on indoor modelling. Reconstructed models submitted by 11 participating teams are evaluated on a dataset comprising 6 point clouds representing indoor environments of different complexity. The evaluation is based on measuring the completeness, correctness, and accuracy of the reconstructed wall elements through comparison with manually generated reference models. The results show that the performance of the methods varies across different datasets, but generally the reconstruction methods achieve better results for the point clouds with higher accuracy and density and fewer gaps, as well as the point clouds representing less complex environments. Filtering clutter points in a pre-processing step contributes to higher correctness, and making strong assumptions on the shape of the reconstructed elements contributes to higher completeness and accuracy for models of Manhattan World environments.

Localization of pedestrians in 3D scene space from single RGB images is critical for various downstream applications. Current monocular approaches employ either the bounding box of pedestrians or the visible parts of their bodies for localization. Both approaches introduce additional error to the location estimation in the case of real-world scenarios – crowded environments with multiple occluded pedestrians. To overcome the limitation, this paper proposes a novel human pose-aware pedestrian localization framework to model poses of occluded pedestrians, where this enables accurate localization in image and ground space. This is done by proposing a light-weight neural network architecture, where this ensures a fast and accurate prediction of missing body parts for downstream applications. Comprehensive experiments on two real-world datasets demonstrate the effectiveness of the framework compared to state-of-the-art in predicting pedestrians missing body parts as well as pedestrian localization.

In historical buildings, surface cracks are important indicators of potential structural damage. Natural disasters and indirect human factors, which are frequently encountered in recent periods, negatively affect historical buildings and structures. Fast and cost-effective crack detection methods play a key role in structural health monitoring of historical buildings. This paper presents methodologies for identifying concrete cracks using deep learning. We propose ReCRNet, a deep learning architecture designed for classifying images of cracks. The performance of the proposed method is evaluated and compared with state-of-the-art methods such as AlexNet, VGG 19, linear support vector machine (SVM), and decision tree (DT). The results show that ReCRNet achieves better performance in terms of accuracy, precision, recall, F1-score, and receiver operating characteristic (ROC), in comparison with the other crack classifiers. Accordingly, the proposed approach is recommended for automatic monitoring of historical buildings and building condition assessments.

The classification of mobile Lidar data is challenged by the complexity of objects in the point clouds and the limited number of available training samples. Incomplete shape, noise points and uneven point density make the extraction of features from point clouds relatively arduous. Additionally, the difference in point density, and size and shape of objects, restricts the utilization of labelled samples from other sources. To solve this problem, we explore the possibility of improving the classification performance of a state-of-the-art deep learning method, Vox-Net, by using auxiliary training samples from a different dataset. We compare the performance of Vox-Net trained with and without the auxiliary dataset. The comparison shows that more instances can be recognized in classes with auxiliary data. At the same time, the performance in classes without complementary data can deteriorate due to the low number of samples in these categories. To achieve a balance in the performance for different categories, we further replace the classification layer of Vox-Net with AdaBoost. The AdaBoost classification displays good recognition ability in classes with few instances but decreases the overall accuracy.

Existing indoor mapping systems have limitations in terms of time efficiency and flexibility in complex environments. While backpack and handheld systems are more flexible and can be used for mapping multi-storey buildings, in some application scenarios, e.g. emergency response, a light-weight indoor mapping eyewear or head-mounted system has practical advantages. In this paper, we investigate the spatial mapping capability of Microsoft Hololens mixed reality eyewear for 3D mapping of large indoor environments. We provide a geometric evaluation of 3D mesh data captured by the Hololens in terms of local precision, coverage, and global correctness in comparison with terrestrial laser scanner data and a reference 3D model. The results indicate the high efficiency and flexibility of Hololens for rapid mapping of relatively large indoor environments with high completeness and centimetre level accuracy.

Building Information Models (BIMs) are of paramount importance in lifecycle management of buildings as they enable collaboration among various stakeholders at different phases of a construction project, from planning to maintenance and operation. However, there is usually inconsistency between the as-is condition of the building and its existing BIM, because BIMs are generally not updated to reflect changes in the environment. Monitoring the changes during a building’s lifecycle and keeping the BIM up-to-date is useful for a variety of applications. Yet this process often involves manual surveying inspections, which are very time-consuming, error-prone, and laborious. In this paper, we present an automated approach for building change detection through a comparison between the BIM and a point cloud of the building indoor environment. The approach is based on point classification and surface coverage to identify discrepancies between the BIM and the point cloud. Experiments on a synthetic dataset and an ISPRS Benchmark dataset show the potential of the proposed approach not only for change detection and identifying discrepancies, but also for locating the removed and new structures of the building in comparison with the BIM. The results are useful for updating the BIM to represent the as-is condition of the building and for temporal analysis of changes during a building’s lifecycle.

Many current indoor localisation approaches need an initial location at the beginning of localisation. The existing visual approaches to indoor localisation perform a 3D reconstruction of the indoor spaces beforehand, for determining this initial location, which is challenging for large indoor spaces. In this research, we present a visual approach for indoor localisation that is eliminating the requirement of any image-based reconstruction of indoor spaces by using a 3D model. A deep Bayesian convolutional neural network is fine-tuned with synthetic images generated from a 3D model to estimate the camera pose of real images. The uncertainty of the estimated camera poses is modelled by sampling the outputs of the Bayesian network fine-tuned with synthetic images. The results of the experiments indicate that a localisation accuracy of 2 metres can be achieved using the proposed approach.

Ultra wide band (UWB) sensors are widely used for indoor positioning; however, in many practical scenarios UWB signals are obscured by people, goods or other obstacles. This results in signal intensity attenuation, multipath effect and even signal loss, which causes a sharp decline in positioning accuracy. Fusion of pedestrian dead-reckoning (PDR) and UWB is an effective method to achieve high-accuracy positioning under non-line of sight conditions. While traditionally Bayesian filters, such as extended Kalman filter (EKF) and particle filter have been used for UWB/PDR fusion, recently Incremental Smoothing has been shown to achieve high accuracy in other application domains. In this paper, incremental Smoothing based on Tukey kernel function is proposed to fuse UWB and PDR data. We compare the performance of Incremental Smoothing with state of the art fusion algorithms based on EKF, and show that the incremental smoothing algorithm can achieve real-time positioning while exhibiting stronger robustness against intermittent noise, continuous noise and continuous interruption abnormality of UWB data.

The ubiquity of cameras built in mobile devices has resulted in a renewed interest in image-based localisation in indoor environments where the global navigation satellite system (GNSS) signals are not available. Existing approaches for indoor localisation using images either require an initial location or need first to perform a 3D reconstruction of the whole environment using structure-from-motion (SfM) methods, which is challenging and time-consuming for large indoor spaces. In this paper, a visual localisation approach is proposed to eliminate the requirement of image-based reconstruction of the indoor environment by using a 3D indoor model. A deep convolutional neural network (DCNN) is fine-tuned using synthetic images obtained from the 3D indoor model to regress the camera pose. Results of the experiments indicate that the proposed approach can be used for indoor localisation in real-time with an accuracy of approximately 2 m.

This article presents an accurate and robust visual indoor localisation approach that not only is infrastructure-free, but also avoids accumulation error by taking advantage of (1) the widespread ubiquity of mobile devices with cameras and (2) the availability of 3D building models for most modern buildings. Localisation is performed by matching image sequences captured by a camera, with a 3D model of the building in a model-based visual tracking framework. Comprehensive evaluation of the approach with a photo-realistic synthetic dataset shows the robustness of the localisation approach under challenging conditions. Additionally, the approach is tested and evaluated on real data captured by a smartphone. The results of the experiments indicate that a localisation accuracy better than 10 cm can be achieved by using this approach. Since localisation errors do not accumulate the proposed approach is suitable for indoor localisation tasks for long periods of time and augmented reality applications, without requiring any local infrastructure. A MATLAB implementation can be found on https://github.com/debaditya-unimelb/BIM-Tracker .