1. RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)
- Author
-
Mu, Yao, Chen, Tianxing, Peng, Shijia, Chen, Zanxin, Gao, Zeyu, Zou, Yude, Lin, Lunkai, Xie, Zhiqiang, and Luo, Ping
- Subjects
Computer Science - Robotics ,Computer Science - Artificial Intelligence ,Computer Science - Computation and Language - Abstract
Effective collaboration of dual-arm robots and their tool use capabilities are increasingly important areas in the advancement of robotics. These skills play a significant role in expanding robots' ability to operate in diverse real-world environments. However, progress is impeded by the scarcity of specialized training data. This paper introduces RoboTwin, a novel benchmark dataset combining real-world teleoperated data with synthetic data from digital twins, designed for dual-arm robotic scenarios. Using the COBOT Magic platform, we have collected diverse data on tool usage and human-robot interaction. We present a innovative approach to creating digital twins using AI-generated content, transforming 2D images into detailed 3D models. Furthermore, we utilize large language models to generate expert-level training data and task-specific pose sequences oriented toward functionality. Our key contributions are: 1) the RoboTwin benchmark dataset, 2) an efficient real-to-simulation pipeline, and 3) the use of language models for automatic expert-level data generation. These advancements are designed to address the shortage of robotic training data, potentially accelerating the development of more capable and versatile robotic systems for a wide range of real-world applications. The project page is available at https://robotwin-benchmark.github.io/early-version/, Comment: Project page: https://robotwin-benchmark.github.io/early-version/
- Published
- 2024