Search

This paper presents a novel approach to fruit detection using deep convolutional neural networks. The aim is to build an accurate, fast and reliable fruit detection system, which is a vital element of an autonomous agricultural robotic platform; it is a key element for fruit yield estimation and automated harvesting. Recent work in deep neural networks has led to the development of a state-of-the-art object detector termed Faster Region-based CNN (Faster R-CNN). We adapt this model, through transfer learning, for the task of fruit detection using imagery obtained from two modalities: colour (RGB) and Near-Infrared (NIR). Early and late fusion methods are explored for combining the multi-modal (RGB and NIR) information. This leads to a novel multi-modal Faster R-CNN model, which achieves state-of-the-art results compared to prior work with the F1 score, which takes into account both precision and recall performances improving from 0 . 807 to 0 . 838 for the detection of sweet pepper. In addition to improved accuracy, this approach is also much quicker to deploy for new fruits, as it requires bounding box annotation rather than pixel-level annotation (annotating bounding boxes is approximately an order of magnitude quicker to perform). The model is retrained to perform the detection of seven fruits, with the entire process taking four hours to annotate and train the new model per fruit.

The application of deep learning in robotics leads to very specific problems and research questions that are typically not addressed by the computer vision and machine learning communities. In this paper we discuss a number of robotics-specific learning, reasoning, and embodiment challenges for deep learning. We explain the need for better evaluation metrics, highlight the importance and unique challenges for deep robotic learning in simulation, and explore the spectrum between purely data-driven and model-driven approaches. We hope this paper provides a motivating overview of important research directions to overcome the current limitations, and helps to fulfill the promising potentials of deep learning in robotics. [ABSTRACT FROM AUTHOR]

Farmers are under growing pressure to intensify production to feed a growing population while managing environmental impact. Robotics has the potential to address these challenges by replacing large complex farm machinery with fleets of small autonomous robots. This article presents our research toward the goal of developing teams of autonomous robots that perform typical farm coverage operations. Making a large fleet of autonomous robots economical requires the use of inexpensive sensors, such as cameras for localization and obstacle avoidance. To this end, we describe a vision-based obstacle detection system that continually adapts to environmental and illumination variations and a visionassisted localization system that can guide a robot along crop rows with a complex appearance. Large fleets of robots will become time-consuming to monitor, control, and resupply. To reduce this burden, we describe a vision-based docking system for autonomously refilling liquid supplies and an interface for controlling multiple robots. [ABSTRACT FROM AUTHOR]

This study presents a segmentation pipeline that fuses colour and depth information to automatically separate objects of interest in video sequences captured from a quadcopter. Many approaches assume that cameras are static with known position, a condition which cannot be preserved in most outdoor robotic applications. In this study, the authors compute depth information and camera positions from a monocular video sequence using structure from motion and use this information as an additional cue to colour for accurate segmentation. The authors model the problem similarly to standard segmentation routines as a Markov random field and perform the segmentation using graph cuts optimisation. Manual intervention is minimised and is only required to determine pixel seeds in the first frame which are then automatically reprojected into the remaining frames of the sequence. The authors also describe an automated method to adjust the relative weights for colour and depth according to their discriminative properties in each frame. Experimental results are presented for two video sequences captured using a quadcopter. The quality of the segmentation is compared to a ground truth and other state-of-the-art methods with consistently accurate results. [ABSTRACT FROM AUTHOR]

This paper presents an approach to building a map from a sparse set of noisy observations, taken from known locations by a sensor with no obvious geometric model. The basic approach is to fit an interpolant to the training data, representing the expected observation, and to assume additive sensor noise. This paper takes a Bayesian view of the problem, maintaining a posterior over interpolants rather than simply the maximum-likelihood interpolant, giving a measure of uncertainty in the map at any point. This is done using a Gaussian process (GP) framework. The approach is validated experimentally both in an indoor office environment and an outdoor urban environment, using observations from an omni-directional camera mounted on a mobile robot. A set of training data is collected from each environment and processed offline to produce a GP model. The robot then uses that model to localize while traversing each environment. [ABSTRACT FROM AUTHOR]

This paper addresses the long-standing problem of feature representation in the natural world for autonomous navigation systems. The proposed representation combines Isomap, which is a nonlinear manifold learning algorithm, with expectation maximization, which is a statistical learning scheme. The representation is computed off-line and results in a compact, nonlinear, non-Gaussian sensor likelihood model. This model can be easily integrated into estimation algorithms for navigation and tracking. The compactness of the model makes it especially attractive for deployment in decentralized sensor networks. Real sensory data from unstructured terrestrial and underwater environments are used to demonstrate the versatility of the computed likelihood model. The experimental results show that this approach can provide consistent models of natural environments to facilitate complex visual tracking and data-association problems. [ABSTRACT FROM AUTHOR]