Space-sharing AR interaction on multiple mobile devices with a depth camera

In this paper, we propose a markerless augmented reality (AR) system that works on multiple mobile devices. The relative positions and orientations of the devices and their individual motions are estimated from 3D information in real space obtained by depth cameras attached to the devices. The system allows multiple users to share the AR space and to interact with the same virtual object. To estimate the relative positions and orientations of the devices, the system generates 2D images by looking down from above at the 3D scene obtained by the depth cameras, performs 2D registration using template matching, and obtains a transformation matrix that transforms the coordinate system of one camera to that of another camera. The motion of a camera is estimated using the ICP algorithm to realize markerless AR. Using the proposed system, we created an application that enables multiple users to interact with the same virtual object.