Real-Time Projection-Based Augmented Reality System for Dynamic Objects in the Performing Arts

This paper describes the case study of applying projection-based augmented reality, especially for dynamic objects in live performing shows, such as plays, dancing, or musicals. Our study aims to project imagery correctly inside the silhouettes of flexible objects, in other words, live actors or the surface of actor’s costumes; the silhouette transforms its own shape frequently. To realize this work, we implemented a special projection system based on the real-time masking technique, that is to say real-time projection-based augmented reality system for dynamic objects in performing arts. We installed the sets on a stage for live performance, and rehearsed particular scenes of a musical. In live performance, using projection-based augmented reality technology enhances technical and theatrical aspects which were not possible with existing video projection techniques. The projected images on the surfaces of actor’s costume could not only express the particular scene of a performance more effectively, but also lead the audience to an extraordinary visual experience.