ARTEMIS: Mixed-Reality Environment for Immersive Surgical Telementoring

The golden hour following a traumatic injury holds the highest likelihood where surgical treatment may prevent mortality and morbidity. However, the increasing occurrence of large-scale disasters, overwhelm local medical systems and patients find themselves without timely access to medical expertise. To respond to this problem medical care experts are exploring telemedicine, which typically attempts to use synchronous audiovisual communication. While current telemedicine approaches have limited ability to impact the physical care required for trauma, AR technology allows medical professionals to see their patients while seeing additional digital information. In this paper we describe ARTEMIS (Augmented Reality Technology to Enable reMote Integrated Surgery), an immersive AR-VR telementoring infrastructure that allows experienced surgeons to remotely aid less experienced medical professionals in the field. ARTEMIS provides Mixed Reality immersive visual aids by tracking a patient in real-time and showing a reconstructed 3D point cloud in a VR environment; expert surgeons can interact with the 3D point cloud representation of the patient, instruct the remote novice through real-time 3D annotations projected in AR on the patient's body, using hand-maneuvers shown in AR through an avatar of the expert surgeon, and by projecting small video clips of specific procedures in the AR space for the novice to follow.