Omnidirectional visual computing: Foundations, challenges, and applications.

Omnidirectional images and videos are widespread due to the popularization of devices for capturing and visualization. Unlike pinhole-based imagery, omnidirectional media lie on the surface of a sphere, have a 360° × 180°field of view, and store the light intensities from an entire environment. Notably, applications involving immersive augmented, mixed, and virtual reality experiences benefit from the 360° content. Although defined on the spherical domain, omnidirectional images are often mapped to a (multi-)planar representation, which results in distorted images and degrades the performance of most traditional visual computing algorithms designed to work on the plane. This paper reviews the spherical camera model, the most common capturing devices, and popular (multi-)planar representations of omnidirectional media. It also approaches the main challenges of omnidirectional visual computing, focusing on the deep learning paradigm, and tackles four visual computing applications that strongly explore the potential of omnidirectional imagery. [Display omitted] • We review the foundations of the spherical camera model. • We review standard 360°capturing devices and format representations. • We discuss the main challenges of 360°visual computing and possible solutions. • We showcase the recent advances on four selected applications that explore 360°media. [ABSTRACT FROM AUTHOR]