Search

Ultra-high definition (UHD) 360 videos encoded in fine quality are typically too large to stream in its entirety over bandwidth (BW)-constrained networks. One popular approach is to interactively extract and send a spatial sub-region corresponding to a viewer’s current field-of-view (FoV) in a head-mounted display (HMD) for more BW-efficient streaming. Due to the non-negligible round-trip-time (RTT) delay between server and client, accurate head movement prediction foretelling a viewer’s future FoVs is essential. In this paper, we cast the head movement prediction task as a sparse directed graph learning problem: three sources of relevant information—collected viewers’ head movement traces, a 360 image saliency map, and a biological human head model—are distilled into a view transition Markov model. Specifically, we formulate a constrained maximum a posteriori (MAP) problem with likelihood and prior terms defined using the three information sources. We solve the MAP problem alternately using a hybrid iterative reweighted least square (IRLS) and Frank-Wolfe (FW) optimization strategy. In each FW iteration, a linear program (LP) is solved, whose runtime is reduced thanks to warm start initialization. Having estimated a Markov model from data, we employ it to optimize a tile-based 360 video streaming system. Extensive experiments show that our head movement prediction scheme noticeably outperformed existing proposals, and our optimized tile-based streaming scheme outperformed competitors in rate-distortion performance. [ABSTRACT FROM AUTHOR]

Multiview applications endow final users with the possibility to freely navigate within 3D scenes with minimum-delay. A real feeling of scene navigation is enabled by transmitting multiple high-quality camera views, which can be used to synthesize additional virtual views to offer a smooth navigation. However, when network resources are limited, not all camera views can be sent at high quality. It is therefore important, yet challenging, to find the right tradeoff between coding artifacts (reducing the quality of camera views) and virtual synthesis artifacts (reducing the number of camera views sent to users). To this aim, we propose an optimal transmission strategy for interactive multiview HTTP adaptive streaming. We propose a problem formulation to select the optimal set of camera views that the client requests for downloading, such that the navigation quality experienced by the user is optimized while the bandwidth constraints are satisfied. We show that our optimization problem is NP-hard, and we therefore develop an optimal solution based on the dynamic programming algorithm with polynomial time complexity. To further simplify the deployment, we present a suboptimal greedy algorithm with effective performance and lower complexity. The proposed controller is evaluated in theoretical and realistic settings characterized by realistic network statistics estimation, buffer management, and server-side representation optimization. Simulation results show significant improvement in terms of navigation quality compared with alternative baseline multiview adaptation logic solutions. [ABSTRACT FROM AUTHOR]

For real-time video streaming systems, the video packets arriving after the display deadline of their frames are considered as late-arrival packets, and typically they are discarded. This will affect the current frame and the following ones due to error propagations. For this reason, in this paper, we propose an approach to exploit the late-arrival and out-of-order packets, which includes two mechanisms. The first mechanism will use these packets to update the reference frames to make them more consistent with the encoder side, and this will eventually reduce the error propagations. The second mechanism will use these packets to increase the chance of successfully decoding the Reed-Solomon (RS) code. In the proposed approach, a sub-GOP based systematic RS code is used and optimized to exploit these packets, where the size of each sub-GOP and the parity packet number for each sub-GOP are optimally tuned, taking into consideration the maximum end-to-end delay, the network conditions, and other system parameters, so as to make the best use of the late-arrival packets and to exploit the out-of-order packets. Finally, the experimental results show the advantage of the proposed approach over other approaches. [ABSTRACT FROM AUTHOR]

Interactive multiview video streaming (IMVS) applications permit to freely navigate within 3D scenes. This endows end-users the immersive and virtual reality experiences. Any additional viewpoint of interest can be virtually synthesized by received texture views with associated depth maps. In this paper, we consider the setup where both the encoded texture and depth bitstreams experience packet losses during network streaming. We investigate two packetization strategies and develop a novel strategy to improve error-resilience capabilities for multiview video plus depth transmission, where texture data and the corresponding depth go into the same packet. We examine the resulting distortion of different packetization schemes and analyze the real difference between them in diverse scenarios. In particular, we take into account: (i) various packet loss patterns, where the effect of a burst loss is identified as an important characteristic; (ii) two packet loss models that characterize different probabilities for each scenario. Experimental results demonstrate that, compared with the competing approach, our proposed packetization scheme increases the visual quality by up to 2.32 dB and 1.55 dB for texture views and synthesized views, respectively. [ABSTRACT FROM AUTHOR]