1. Multimodal Unknown Surface Material Classification and Its Application to Physical Reasoning
- Author
-
Jingyi Hu, Junhang Wei, Cui Shaowei, Peng Hao, Shuo Wang, and Zheng Lou
- Subjects
Surface (mathematics) ,Network architecture ,Normal force ,Computer science ,business.industry ,Deep learning ,Pattern recognition ,Object (computer science) ,Convolutional neural network ,Computer Science Applications ,Acceleration ,Control and Systems Engineering ,Robot ,Artificial intelligence ,Electrical and Electronic Engineering ,business ,Information Systems - Abstract
Unknown surface material classification can inform a robot about material properties, enabling it to interact with environments appropriately. Recent research has leveraged multimodal data using deep learning to improve the performance of surface material classification. In this paper, we present a deep learning model, multimodal temporal convolutional neural network (MTCNN), which integrates energy spectrum, dilated convolutions, and sequence poolings into a unified network architecture. The proposed model can learn material representations from auditory and multi-tactile (i.e., acceleration, normal force, and friction force) data generated by dragging a tool along surfaces, and distinguish unknown object surface materials into categories. For surface material data collection, a tool is also designed to detect different object surfaces. The performance of MTCNN is evaluated on a public dataset and the highest classification accuracy is 87.55%. A robotic curling example is provided to illustrate how the presented model helps the robot in manipulation.
- Published
- 2022