Back to Search
Start Over
Quality assessment for view synthesis using low-level and mid-level structural representation
- Source :
- Signal Processing: Image Communication, Signal Processing: Image Communication, Elsevier, 2019, 74, pp.309-321. ⟨10.1016/j.image.2019.03.005⟩
- Publication Year :
- 2019
- Publisher :
- HAL CCSD, 2019.
-
Abstract
- View synthesis is the most important technique in multi-view and free-viewpoint videos. The whole view synthesis includes the acquisition and processing of texture and depth images, and the virtual view rendering stage. Existing quality metrics for view synthesis have limited ability for the whole synthesis process for the following reasons. First, they are dedicated to a single stage of view synthesis, overlooking the commonality of all the possible distortions introduced in the whole process. Moreover, they only extract low-level features for quality assessment, ignoring the perceptual degradation caused by the mid-level contours that are destructed by heavy distortions in texture/depth images and the imperfect view rendering, which represent the spatial distribution/connection of adjacent contour pixels. Inspired by the above facts, this paper presents a quality metric for view synthesis using both Low-level and Mid-level Structural representation (LMS), aiming to accurately evaluate the distortions in the whole view synthesis process. Specifically, the scale space is first constructed to mimic the hierarchical property of the human visual system. Then, the statistics of gradient orientation is integrated with the statistics of gradient intensity for the low-level structural representation, which is motivated by the importance of the orientation selectivity mechanism to visual perception. Further, the mid-level structure is represented using bag of words for contour description based on the sparse coding of the primary visual cortex. Then the distances of both the low-level and mid-level features between the synthesized and reference images are calculated. Finally, two distances are integrated to generate the whole quality score. Extensive experiments on two public view synthesis databases demonstrate the superiority of the proposed method to the state-of-the-arts in evaluating the quality of the whole view synthesis.
- Subjects :
- Visual perception
Pixel
business.industry
Computer science
ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION
020206 networking & telecommunications
Pattern recognition
02 engineering and technology
Rendering (computer graphics)
View synthesis
Scale space
Bag-of-words model
[INFO.INFO-TI]Computer Science [cs]/Image Processing [eess.IV]
Signal Processing
Human visual system model
0202 electrical engineering, electronic engineering, information engineering
020201 artificial intelligence & image processing
Computer Vision and Pattern Recognition
Artificial intelligence
Electrical and Electronic Engineering
Neural coding
business
Software
ComputingMilieux_MISCELLANEOUS
Subjects
Details
- Language :
- English
- ISSN :
- 09235965 and 18792677
- Database :
- OpenAIRE
- Journal :
- Signal Processing: Image Communication, Signal Processing: Image Communication, Elsevier, 2019, 74, pp.309-321. ⟨10.1016/j.image.2019.03.005⟩
- Accession number :
- edsair.doi.dedup.....6a79d5cce39a308ec23dc1a8ed9fe9a2