Back to Search Start Over

Unseen Object Instance Segmentation with Fully Test-time RGB-D Embeddings Adaptation

Authors :
Zhang, Lu
Zhang, Siqi
Yang, Xu
Qiao, Hong
Liu, Zhiyong
Publication Year :
2022

Abstract

Segmenting unseen objects is a crucial ability for the robot since it may encounter new environments during the operation. Recently, a popular solution is leveraging RGB-D features of large-scale synthetic data and directly applying the model to unseen real-world scenarios. However, the domain shift caused by the sim2real gap is inevitable, posing a crucial challenge to the segmentation model. In this paper, we emphasize the adaptation process across sim2real domains and model it as a learning problem on the BatchNorm parameters of a simulation-trained model. Specifically, we propose a novel non-parametric entropy objective, which formulates the learning objective for the test-time adaptation in an open-world manner. Then, a cross-modality knowledge distillation objective is further designed to encourage the test-time knowledge transfer for feature enhancement. Our approach can be efficiently implemented with only test images, without requiring annotations or revisiting the large-scale synthetic training data. Besides significant time savings, the proposed method consistently improves segmentation results on the overlap and boundary metrics, achieving state-of-the-art performance on unseen object instance segmentation.<br />Comment: Accepted to ICRA 2023

Details

Database :
arXiv
Publication Type :
Report
Accession number :
edsarx.2204.09847
Document Type :
Working Paper