Search

Inspired by recent promising results in sim-to-real transfer in deep learning we built a realistic simulation environment combining a Robot Operating System (ROS)-compatible physics simulator (Gazebo) with Cycles, the realistic production rendering engine from Blender. The proposed simulator pipeline allows us to simulate near-realistic RGB-D images. To showcase the capabilities of the simulator pipeline we propose a case study that focuses on indoor robotic farming. We developed a solution for sweet pepper yield estimation task. Our approach to yield estimation starts with aerial robotics control and trajectory planning, combined with deep learning-based pepper detection, and a clustering approach for counting fruit. The results of this case study show that we can combine real time dynamic simulation with near realistic rendering capabilities to simulate complex robotic systems., Comment: Submitted to ICUAS 2022. Under review

In this paper we present our hardware design and control approaches for a mobile manipulation platform used in Challenge 2 of the MBZIRC 2020 competition. In this challenge, a team of UAVs and a single UGV collaborate in an autonomous, wall-building scenario, motivated by construction automation and large-scale robotic 3D printing. The robots must be able, autonomously, to detect, manipulate, and transport bricks in an unstructured, outdoor environment. Our control approach is based on a state machine that dictates which controllers are active at each stage of the Challenge. In the first stage our UGV uses visual servoing and local controllers to approach the target object without considering its orientation. The second stage consists of detecting the object's global pose using OpenCV-based processing of RGB-D image and point-cloud data, and calculating an alignment goal within a global map. The map is built with Google Cartographer and is based on onboard LIDAR, IMU, and GPS data. Motion control in the second stage is realized using the ROS Move Base package with Time-Elastic Band trajectory optimization. Visual servo algorithms guide the vehicle in local object-approach movement and the arm in manipulating bricks. To ensure a stable grasp of the brick's magnetic patch, we developed a passively-compliant, electromagnetic gripper with tactile feedback. Our fully-autonomous UGV performed well in Challenge 2 and in post-competition evaluations of its brick pick-and-place algorithms.

In this paper we propose a robotic system for Irrigation Water Management (IWM) in a structured robotic greenhouse environment. A commercially available robotic manipulator is equipped with an RGB-D camera and a soil moisture sensor. The two are used to automate the procedure known as "feel and appearance method", which is a way of monitoring soil moisture to determine when to irrigate and how much water to apply. We develop a compliant force control framework that enables the robot to insert the soil moisture sensor in the sensitive plant root zone of the soil, without harming the plant. RGB-D camera is used to roughly estimate the soil surface, in order to plan the soil sampling approach. Used together with the developed adaptive force control algorithm, the camera enables the robot to sample the soil without knowing the exact soil stiffness a priori. Finally, we postulate a deep learning based approach to utilize the camera to visually assess the soil health and moisture content.