Search

The paper reports a new view synthesis method for omnidirectional video with the ability to slightly displace a virtual viewpoint, i.e. the paper describes a novel synthesis method for 3DoF+ 360 video. This new method is noteworthy because of its high versatility and reliability: the method is appropriate for both perspective and omnidirectional input views, is able to render both perspective and omnidirectional views, and the produced synthetic views differ from the respective ground truth images less than with other view synthesis methods. These important features result from several innovations, e.g., prioritization of input views, efficient inpainting adapted to equirectangular projections, and efficient color correction. The experimental results demonstrate that the new method outperforms the state-of-the-art methods used hitherto.

Delays in the delivery of immersive video to Head Mounted Devices (HMDs) are considered in the paper. The goal of this paper is to analyze the impact of video streaming latency on user experience with HMDs. The paper reports the results of the subjective quality assessment as a function of delay. The possible practical solutions are identified and, on this base, an experimental model of the considered systems is proposed. In order to properly test the influence of the streaming delay on the quality, the extensive subjective tests have been performed. Interesting conclusions have been drawn, showing that human acceptance of delay of translation of virtual point of viewing is much stronger than for delay of rotation of virtual viewing direction in immersive video. Finally, the influence of such observations on the system architectures is concluded.

Payment channels emerged recently as an efficient method for performing cheap micropayments in cryptocurrencies. In contrast to traditional on-chain transactions, payment channels have the advantage that they allow for nearly unlimited number of transactions between parties without involving the blockchain. In this work, we introduce Perun, an off-chain channel system that offers a new method for connecting channels that is more efficient than the existing technique of ``routing transactions'' over multiple channels. To this end, Perun introduces a technique called ``virtual payment channels'' that avoids involvement of the intermediary for each individual payment. In this paper we formally model and prove security of this technique in the case of one intermediary, who can be viewed as a ``payment hub'' that has direct channels with several parties. Our scheme works over any cryptocurrency that provides Turing-complete smart contracts. As a proof of concept, we implemented Perun's smart contracts in Ethereum.

The paper deals with the problem of the optimal selection of the real views used for the virtual view synthesis in the free navigation systems. The practical free navigation system requires the real-time view synthesis. Therefore, the virtual view cannot be synthesized using the information from all the real cameras, but only few, most proper ones. In the paper the method of fast selection of two proper real views is presented. The proposed approach allows to select the real views for different camera arrangements, thus can be applied to the practical free navigation systems.

This paper presents the fast virtual view synthesis method for free navigation purposes. Presented method was implemented on CPU in order to allow free navigation for even cheap personal computers with integrated graphic cards. The described synthesis technique is based on the hybrid view synthesis, combining advantages of forward-type and backward-type synthesis. It allows to synthesize high resolution sequences in the real time, allowing the viewer to freely navigate around the scene, e.g. volleyball playfield. Three implementations were developed and tested: single-threaded, vectorized and multithreaded implementation.

In this paper we introduce an algorithm for omnidirectional virtual view synthesis based on Omnidirectional Video plus Depth (OVD) format. The implementation is done on the basis of View Synthesis Reference Software (VSRS) developed by MPEG of ISO/IEC. Also, we address the problem of lack of benchmark data disallowing objective quality assessment. We present a method for generating test images in OVD representation along with example omnidirectional images and omnidirectional depths, called “Poznan Hall 360 ”.

The paper reports the results on virtual navigation in a sports hall. The video data is gathered and processed off-line thus producing the multiview plus depth representation of a scene. This representation is an input to the rendering server. Even a typical PC computer is able to serve a number of users that freely and independently navigate in real-time and watch virtual video. The users are able to navigate in the scene changing the virtual viewpoint and virtual view direction. The service is provided through a webpage, so a viewer does not need to install any software on smartphone, tablet or notebook. The paper discusses the technical solutions together with the rationale for their choice.

In the paper an adaptive color correction method for virtual view synthesis is presented. It deals with the typical problem in free navigation systems – different illumination in views captured by different cameras acquiring the scene. The proposed technique adjusts the local color characteristics of objects visible in two real views. That approach allows to significantly reduce number and visibility of color artifacts in the virtual view. Proposed method was tested on 12 multiview test sequences. Obtained and presented in the paper results show, that proposed color correction provides increase of the virtual view quality measured by PSNR, SSIM and subjective evaluation.

The demonstration is aimed at presenting a simple free-viewpoint television system that is under development at Poznan University of Technology, Poznan, Poland. Video acquisition using sparsely distributed pairs of video cameras, the depth estimation from video, the virtual video rendering on a server feature the perspective FTV system. The original depth estimation and view synthesis algorithms ensure good quality of the virtual views generated during virtual walks around a scene. The system is developed for broadcasting sport (e.g. judo, karate, volleyball) and cultural (e.g. amateur or professional theater performances) events, as well as for interactive courses and manuals (medical, cosmetics, dancing, technical etc.).

This paper presents a new depth estimation method for multiview systems with arbitrary camera locations. The method exploits the graph cuts method, where vertices of the graph represent segments used for controlling the trade-off between the quality of depth maps and the time of estimation, while preserving the original resolution of a depth map. Moreover, the inter-view consistency of the depth maps, crucial for free-viewpoint television systems, is ensured by introduction of suitable connections in the optimized graph. It makes the proposed method the first that allows generation of spatially-consistent multiview depth maps using segmentation-based estimation. A new method of the adaptive calculation of the smoothing coefficient was also presented. The performance of the proposed algorithm was tested and compared with the state-of-the-art DERS method, showing an significant improvement, both in terms of the depth maps fidelity and the time of estimation.

This paper presents the temporal enhancement of the graph-based depth estimation method, designed for multiview systems with arbitrarily located cameras. The primary goal of the proposed enhancement is to increase the quality of estimated depth maps and simultaneously decrease the time of estimation. The method consists of two stages: the temporal enhancement of segmentation required in used depth estimation method, and the exploitation of depth information from the previous frame in the energy function minimization. Performed experiments show that for all tested sequences the quality of estimated depth maps was increased. Even if only one cycle of optimization is used in proposed method, the quality is higher than for unmodified method, apart from number of cycles. Therefore, use of proposed enhancement allows estimating depth of better quality even with 40% reduction of estimation time.

In the article we deal with the problem of camera positioning in sparse multiview systems with applications to free navigation. The limited number of the cameras, though makes the system relatively practical, implies problems with proper depth estimation and virtual view synthesis, due to increased amount of the occluded areas. We present experimental results for the optimal positioning of the cameras, depending on two factors — characteristics of an acquired scene and the multi-camera system (linear or circular camera setup). The results show the correlation between the number of occlusions in the scene and a gain from using camera pairs instead of uniformly distributed cameras.

In free-viewpoint television, high-quality depth maps are substantial for a virtual view synthesis for free navigation purposes. In this paper, we propose a new method of depth map quality improvement and resolution increase. Our method is intended for low resolution depth maps acquired through estimation using multiview video. The proposed depth map quality improvement is based on segmentation of an acquired high-resolution image. In the estimated depth map, searching for outliers is performed in the neighborhood created from similar segments in the acquired view. Experimental results show high effectiveness of the presented method of depth maps refinement, especially for object edges.

The paper presents the concept of a practical free-viewpoint television system with purely optical depth estimation. The system consists of camera modules that contain pairs or triples of cameras together with the respective microphones. The camera modules can be sparsely located in arbitrary positions around a scene. Each camera module is equivalent to a video camera with a depth sensor and microphones. The hardware requirements, the video and audio processing algorithms and the preliminary experimental results are reported. In particular, for such systems, a compression technique is discussed that is more efficient than the new 3D-HEVC technology. A set of new test sequences obtained with the use of camera pairs are presented.

The paper considers the influence of a lossy compression of multiview video on the quality of estimated depth maps. We have encoded a set of views with commonly known compression techniques used in compact IP cameras: HEVC (MPEG H part 2 / H.265), A VC (MPEG 4 part 10 / H.264) and MPEG 2 part 2 / H.262. Basing on the decoded data we have estimated depth maps and rendered a set of the intermediate virtual views. A drop of the quality was studied by comparison of virtual views rendered out of the depth maps estimated from an uncompressed video. The obtained results show that with a reasonable compression rate, lower quality of virtual views is acceptable for real implementable FTV systems.

In the paper, we propose a new method for the virtual view synthesis called as Multiview Synthesis. In our approach, virtual views are synthesized using two neighboring real views, but the disoccluded areas are not inpainted, but filled by the information from the further real views — additional views, traditionally not included in the view synthesis. The whole synthesis is performed with triangles rather than with individual pixels. In the proposed Multiview Synthesis, additional steps of adaptive color correction, blurred edge removal and spatial edge blurring are also included. The experimental results show that both the objective and subjective quality of the synthesized views is significantly higher than the quality of views obtained using the state-of-the-art MPEG reference view synthesis software.

We deal with the processing of multiview video acquired by the use of practical thus relatively simple acquisition systems that have a limited number of cameras located around a scene on independent tripods. The real-camera locations are nearly arbitrary as it would be required in the real-world Free-Viewpoint Television systems. The appropriate test video sequences are also reported. We describe a family of original extensions and adaptations of the multiview video processing algorithms adapted to arbitrary camera positions around a scene. The techniques constitute the video processing chain for Free-Viewpoint Television as they are aimed at estimating the parameters of such a multi-camera system, video correction, depth estimation and virtual view synthesis. Moreover, we demonstrate the need for new compression technology capable of efficient compression of sparse convergent views. The experimental results for processing the proposed test sequences are reported.

Service oriented architectures (SOAs) are rapidly becoming an accepted means of providing network information exchange across a heterogeneous fabric of nodes. Given the complexity of such architectures and multiple levels of system interactions, creating a valid and usable SoS model for SOA application using a single technique that captures the desired level of details can be a daunting task. In this paper we give details of a hybrid approach to modeling and simulation of a specific SOA that is named MCSOA and has been configured for potential defense applications. In this approach, at the lowest system level (call it white-box level) we are using colored Petri nets (CPNs) to model internal protocols, communications, and resource consumption. This allows us to validate the components of the system and identify weaknesses at the component level. For the interaction at the highest system level (call it black-box level), we are using a set of discrete-event simulation tools known as MESA/Extend. The major advantage of this combination is that the assumptions at the black-box level are fully validated at the white-box level, which greatly simplifies and accelerates the design, development, and testing of the simulations of very large scale SOA-based systems of systems.