Search

Certain telerobotic applications, including telerobotics in space, pose particularly demanding challenges to both technology and humans. Traditional bilateral telemanipulation approaches often cannot be used in such applications due to technical and physical limitations such as long and varying delays, packet loss, and limited bandwidth, as well as high reliability, precision, and task duration requirements. In order to close this gap, we research model-augmented haptic telemanipulation (MATM) that uses two kinds of models: a remote model that enables shared autonomous functionality of the teleoperated robot, and a local model that aims to generate assistive augmented haptic feedback for the human operator. Several technological methods that form the backbone of the MATM approach have already been successfully demonstrated in accomplished telerobotic space missions. On this basis, we have applied our approach in more recent research to applications in the fields of orbital robotics, telesurgery, caregiving, and telenavigation. In the course of this work, we have advanced specific aspects of the approach that were of particular importance for each respective application, especially shared autonomy, and haptic augmentation. This overview paper discusses the MATM approach in detail, presents the latest research results of the various technologies encompassed within this approach, provides a retrospective of DLR's telerobotic space missions, demonstrates the broad application potential of MATM based on the aforementioned use cases, and outlines lessons learned and open challenges.

The MMX - Martian Moons eXploration - mission, as the name already suggests, aims to explore the two moons of Mars, Phobos and Deimos. The goal of this space mission led by JAXA is to acquire the scientific data necessary to understand the composition, structure, and history of these peculiar celestial bodies. The first man-made object to ever land on the larger and closer of these two moons, Phobos, shall be the small and lightweight MMX rover. The rover will be designed, manufactured and operated jointly by CNES and DLR. After separation from the carrier spacecraft, landing, uprighting, and deployment, this rover will start to drive in the low-gravity environment of Phobos surface and perform scientific operations. The rover will have a length of 44cm and a weight below 30kg. It will be solar-powered to operate for an intended mission duration of 90 days [1]. Communication round-trip times between Earth and Mars are already eight to forty minutes. For Phobos, we expect significantly higher values due to the need for relay satellites and limited communication windows. This leads to a requirement for a high level of autonomy for the robot, particularly its navigation capabilities, in order to maximize the scientific output of the mission. We will thus develop a navigation solution and integrate it as a software component running on the MMX rovers on-board computer. This navigation software will be verified in the introductory commissioning phase and is intended to be useful in the subsequent main operations phase. On the one hand, our design is inspired by the previous successful NASA planetary rovers, which have been the pioneers of this software technology. On the other hand, the design of the MMX rover brings its own sets of specifics and limitations to the table, and the uncharted celestial body Phobos itself is a source of several major and unprecedented challenges. For some of these, we can build upon the experience gained with the MASCOT mobile asteroid lander [2], which was deployed on the asteroid Ryugu in 2018 and successfully performed jumps for relocalization in the micro-gravity environment. The most notable challenges provided by the MMX rover design are the navigation cameras located at a fixed position and orientation w.r.t. the rovers body at a height of only 30cm above ground; the skid-steered locomotion of the rover limiting its turning speed; and restrictions on weight and power consumption further limiting the range of permissible operations. The most notable challenges provided by Phobos include not knowing the map of the terrain at the operational scale beforehand - the best maps available are at a resolution of 5m/px; the unknown soil composition; and the unknown local gravity. The combination of the rover and the celestial body is also a source of challenges: the behavior of wheels in contact with the soil is impossible to investigate beforehand, apart from software simulations, and Phobos very fast rotation - a Phobos day only lasts eight Earth hours - in the combination with the very slow maximum rover speed of approximately 4mm/s, makes shadows move relatively quickly, which could confuse visual odometry approaches. The on-board computer provides its own set of limitations and pre-requisites such as memory allocation and orchestration of concurrently running software processes. In our workshop contribution, we will identify and categorize challenges for navigation on Phobos and sketch our planned solutions to tackle them. Our planned navigation architecture will contain several FPGA and CPU-based modules: Dense depth data will be computed via Semi-Global Matching [3] on a FPGA. This is the basis for a stereo visual odometry such as [4] used to estimate the robot's trajectory. Further modules include an obstacle classification on individual depth images similar to [5] and possibly further mapping modules to create maps in compact representations to be sent to operators on Earth. Such obstacle and map information can then be used to realize autonomous emergency stop behavior up to future reactive obstacle avoidance or path planning modules to support (semi-)autonomous operation. These developments are based on experience we gained developing complex autonomous robotic navigation systems [6, 7] that we tested and evaluated in several field tests at Moon-analogue environments on the volcano Mt. Etna, Sicily, Italy [8, 9]. Some of the greatest challenges arise from the daring ambition to bring a rover into an environment where mankind has never been before, and expecting it to drive there to some extent autonomously. But that is also what makes this mission interesting in the first place. The scientific and technological discussions at the workshop may both help us to steer our decision making and enrich the scientific community with our findings. References: [1] J. Bertrand, et al., Roving on Phobos: Challenges of the MMX Rover for Space Robotics, ASTRA (2019) [2] J. Reill et al., MASCOT - Asteroid Lander with Innovative Mobility Mechanism, ASTRA (2015) [3] H. Hirschmüller, Stereo processing by semiglobal matching and mutual information, TPAMI (2007) [4] H. Hirschmüller, et al., Fast, unconstrained camera motion estimation from stereo without tracking and robust statistics, ICARCV (2002) [5] C. Brand, et al., Stereo-Vision Based Obstacle Mapping for Indoor / Outdoor SLAM, IROS (2014) [6] M. J. Schuster, et al., Towards Autonomous Planetary Exploration: The Lightweight Rover Unit (LRU), its Success in the SpaceBotCamp Challenge, and Beyond, JINT (2017) [7] M. J. Schuster, et al., Distributed stereo vision-based 6D localization and mapping for multi-robot teams, JFR (2018) [8] M. Vayugundla, et al., Datasets of Long Range Navigation Experiments in a Moon Analogue Environment on Mount Etna, ISR (2018) [9] A. Wedler, et al., First Results of the ROBEX Analogue Mission Campaign: Robotic Deployment of Seismic Networks for Future Lunar Missions, IAC (2017)

Intelligent robotic coworkers are considered a valuable addition in many application areas. This applies not only to terrestrial domains, but also to the exploration of our solar system. As humankind moves toward an ever increasing presence in space, infrastructure has to be constructed and maintained on distant planets such as Mars. AI-enabled robots will play a major role in this scenario. The space agencies envisage robotic co-workers to be deployed to set-up habitats, energy, and return vessels for future human scientists. By leveraging AI planning methods, this vision has already become one step closer to reality. In the METERON SUPVIS Justin experiment, the intelligent robotic coworker Rollin’ Justin was controlled from Astronauts aboard the International Space Station (ISS) in order to maintain a Martian mock-up solar panel farm located on Earth to demonstrate the technology readiness of the developed methods. For this work, the system is demonstrated at AAAI 2019, controlling Rollin’ Justin located in Munich, Germany from Honolulu, Hawaii.