Search

3DTV promises to become the mainstream of next-generation TV systems. Highresolution 3DTV provides users with a vivid watching experience. Moreover, free-viewpoint view synthesis (FVVS) extends the common two-view stereo 3D vision into virtual reality by generating unlimited views from any desired viewpoint. In the next-generation 3DTV systems, the set-top box (STB) SoC requires both a high-definition (HD) multiview video-coding (MVC) decoder to reconstruct the real camera-captured scenes and a free-viewpoint view synthesizer to generate the virtual scenes [1–2].

Fractional motion estimation (FME) is widely used in video compression standards. In H.264/AVC, the precision of motion vector is down to quarter pixels to improve the coding efficiency. However, FME occupies over 45% of the computation complexity in an H.264 encoder and this high complexity limits the processing capability. In this paper, a single-iteration full search FME is proposed. By the algorithm and architecture co-optimization, the bandwidth to the frame buffer is reduced by 31%. Furthermore, 82% of circuit area for the Hadamard transformation and subtraction are saved from the direct implementation. Compared with prior arts, the proposed design supports 3.39 × higher throughput with only 0.02 dB PSNR drop. Thus, the specification of 4096 × 2160 quad full high definition H.264/AVC FME processing can be achieved.

Multiview video can bring the 3D and virtual reality perceptual experience to users by its multiple view point characteristic. To support the smooth and free view point switching, both matrix-based depth image based rendering and the complex virtual view interpolation schemes are required. In order to reduce the high computation complexity and avoid the iterative processing scheduling in conventional view interpolation flows, a single iteration view interpolation algorithm is proposed. By use of the proposed algorithm, the redundant warping operations are reduced by 86%. In addition, based on the proposed artifact detecting and removing algorithm, artifacts due to imperfect depth maps can be detected and eliminated at the same time. Therefore, no additional post-processing or iteration is required and the single iteration processing is achieved.

Multiview video coding (MVC) is getting more and more attention in recent years because of the realistic perception it can bring. However, the current H.264-based coding scheme cannot get enough coding gain in the rate-distortion (R-D) performance. Furthermore, the view synthesis and random access between views are also design challenges. In this paper, a multiview hybrid coding system with texture-depth synthesis is proposed. By sharing the prediction information between texture and depth maps, the depth map can be reconstructed with reduced bitrate overhead. Then, after using matrix-based warping and adaptive disparity compensation, the R-D performance of the proposed system can be about 2 to 4 dB higher than the current MVC coding scheme at low bitrate range.

3-D video will become one of the most important video technologies in the next generation of television. Due to ultra high data bandwidth requirement for 3-D video, effective compression technology becomes an essential part in the infrastructure. Thus stereo and multiview video coding (MVC) plays a critical role. However, MVC systems require much more computational complexity relative to mono-view video coding systems. Therefore, an efficient prediction scheme is necessary for encoding. In this paper, a new fast motion estimation (ME) algorithm is proposed. By utilizing disparity estimation (DE) to find corresponding blocks between different views, the coding information such as motion vectors can be effectively shared and reused from the coded view channel. Therefore, the computation for ME in most view channels can be greatly reduced. Experimental results show that compared with the full search block matching algorithm applied to both ME and DE, the proposed algorithm saves 95% computation with near-FSBMA quality.

This paper presented a novel depth map generation method - the short-term motion assisted color segmentation, which combines the pictorial, monocular and binocular depth cues of human vision. The proposed method utilizes a motion/edge registration technique to avoid the motion jitter error in common motion segmentation. And the motion/image segment adaptation algorithm matches the connected components with the motion segments. Even for static scene, the connected component algorithm is still working for the depth map generation. The experimental results show that the adaptation of motion and image segmentation improves quality and smoothness of the depth map both in the spatial and temporal domain.

Multiview video coding (MVC) plays a critical role in 3D-TV systems. However, MVC require much more memory bandwidth and computational complexity relative to mono-view video coding. Therefore, an efficient prediction scheme is necessary for computation reduction. In this paper, a new fast prediction algorithm, computation-free motion estimation (ME) with inter-view mode decision, is proposed. By utilizing disparity estimation to find corresponding blocks between different views, the coding information, such as modes and motion vectors, can be effectively reused from the coded view channel. Therefore, the computation for ME can be almost totally neglected in most view channels. Besides, the proposed algorithm also provides the computation scalability. Experimental results show that compared with the full search block matching algorithm (FSBMA), the proposed algorithm not only save 61-72% computation with near-FSBMA quality, but save almost 100% computation with a little quality drop of 0.23-0.36 dB. Thus, computation-free ME is achieved.

Luminance and chrominance correction (LCC) is important in multi-view video coding (MVC) because it provides better rate-distortion performance when encoding video sequences captured by ill-calibrated multi-view cameras. This paper presents a robust and fast LCC algorithm based on motion compensated linear regression which reuses the motion information from the encoder. We adopt the linear weighted prediction model in H.264/AVC as our LCC model. In our experiments, the proposed LCC algorithm outperforms basic histogram matching method up to 0.4dB with only few computational overhead and zero external memory bandwidth. So, the dataflow of this method is suitable for low bandwidth/low power VLSI design for future multi-view applications.

An efficient depth image based rendering with edge dependent depth filter and interpolation is proposed. The proposed method can solve the hole-filling problem in DIBR system efficiently with high quality. The PSNR of the proposed method is better than the previous work by 6 dB and the subjective view shows the quality is better. In addition to that, the number of instruction cycles is 3.7 percent compared with the previous work

Stereo video systems require double bandwidth and more than twice the computational complexity relative to mono-video systems. Thus, an efficient coding scheme is necessary for transmitting stereo video. We propose a novel stereo video coding system by exploiting a joint prediction scheme which combines three prediction schemes to achieve high coding efficiency and low computational complexity. Joint block compensation improves the visual quality. Motion vector prediction and mode pre-decision utilize the features of stereo video to reduce the computational complexity with up to 8-9 times acceleration. Experiments show that the proposed joint prediction scheme is 2 dB better than MPEG-4 TS and 3 dB better than MPEG-4 SP.

An efficient coding system is required for immersive teleconference to transmit stereo video. We propose a novel stereo video coding system by exploiting mesh-based disparity estimation and compensation scheme to achieve high coding efficiency and view synthesis ability. Based on the base-layer-enhance-layer structure, this system can provide stereoscopic scalability and compatibility to standards. With considering asymmetric spatial resolution property, good subjective quality can be achieved in ultra low bitrate situation. A novel fast disparity estimation algorithm named as two-stage iterative block and octagonal matching (TS-IBOM) algorithm is also proposed for this system. Experiments show that the proposed disparity estimation algorithm can generate accurate disparity vectors quickly. It is also shown that the proposed coding system has better coding efficiency than MPEG-4 simple profile and temporal scalability tool. The ultra low bitrate of 69 Kbps can be reached to encode 384/spl times/192 60 fps stereo video.

Disparity estimation is a very important operation in stereo image and video compression. However, existing disparity estimation algorithms require too large computation power. In this paper, we propose a fast disparity estimation algorithm for mesh-based stereo image and video compression with two-stage hybrid approach, which is names as Two Stage Iterative Block and Octagonal Matching algorithm (TS-IBOM). The first stage of this algorithm is Iterative Block Matching algorithm, which can give a good initial guess of the disparity vectors with a little computation. In the second stage, Iterative Octagonal Matching algorithm is employed to refine the disparity vectors. A local updating scheme is also proposed to further accelerate the process by skipping nodes whose disparity vectors have been derived already. Experimental results show that the proposed algorithm can give good results and is 18 times faster than Octagonal Matching. This algorithm is suitable to be employed in mesh-based stereo image and video compression systems.