Search

Virtual reality systems use digital models to provide interactive viewing. We present a 3D digital video system that attempts to provide the same capabilities for actual performances such as dancing. Recreating the original dynamic scene in 3D, the system allows photorealistic interactive playback from arbitrary viewpoints using video streams of a given scene from multiple perspectives.

We describe an infrastructure for multiple perspective interactive video (MPI-Video) systems. An MPI-Video system assimilates data from multiple disparate sensors, allowing a user to interact with them in a meaningful way. At the heart of an MPI-Video system is an environment model (EM) that both assimilates data from multiple sensors and acts as an information server to clients. A collection of strongly-coupled dynamic state estimation algorithms does the assimilation. Results show that our infrastructure is easy to configure, easy to extend, and is effective at modeling a dynamic environment.

Conventional video limits the viewers to a fixed predetermined viewpoint. With the advancement of image generation and analysis techniques, it becomes possible to overcome this limitation. We present a framework for the generation of 3D video, based on the uses of multiple cameras from different perspectives. Exploiting a priori knowledge of the scene geometry and dynamic objects, we can reconstruct 3D shapes of objects and generate virtual views of spatio-temporal events. An overview of the framework is first presented. Then we report our ongoing research to overcome issues in segmentation and playback.