Search

International audience; In this paper, we present a no-reference (NR) quality predictor for stereoscopic/3D images based on statistics aggregation of monocular and binocular local contrast features. In particular, for left and right views, we first extract statistical features of the image gradient magnitude (GM) and the Laplacian of Gaussian (LoG), describing the image local structures from different perspectives. The monocular statistical features are then combined to derive the binocular features based on a linear summation model using weightings based on LoGresponse and image local-entropy, independently. These weights can effectively simulate the strength of the views dominance on binocular rivalry (BR) behavior of the human visual system. Subsequently, we further compute the GM features of the difference map between left and right views reflecting the distortion on disparity/depth information. Finally, the BR-inspired combined monocular and disparityrelated binocular features associated with subjective quality scores are jointly used to construct a learned regression model relying on support vector machine regressor. Experimental results on three 3D-IQA benchmark databases demonstrate that our method achieves high quality prediction accuracy and competitive performance compared to state-of-the-art methods.

International audience; In this paper, we propose a saliency-weighted stereoscopic JND (SSJND) model constructed based on psychophysical experiments, accounting for binocular disparity and spatial masking effects of the human visual system (HVS). Specifically, a disparity-aware binocular JND model is first developed using psychophysical data, and then is employed to estimate the JND threshold for non-occluded pixel (NOP). In addition, to derive a reliable 3D-JND prediction, we determine the visibility threshold for occluded pixel (OP) by including a robust 2D-JND model. Finally, SSJND thresholds of one view are obtained by weighting the resulting JND for NOP and OP with their visual saliency. Based on subjective experiments, we demonstrate that the proposed model outperforms the other 3D-JND models in terms of perceptual quality at the same noise level.

Just noticeable difference (JND) for stereoscopic 3D content reflects the maximum tolerable distortion; it corresponds to the visibility threshold of the asymmetric distortions in the left and right contents. The 3D-JND models can be used to improve the efficiency of the 3D compression or the 3D quality assessment. Compared to 2D-JND models, the 3D-JND models appeared recently and the related literature is rather limited. In this paper, we give a deep and comprehensive study of the pixel-based 3D-JND models. To our best knowledge, this is the first review on 3D-JND models. Each model is briefly described by giving its rationale and main components in addition to providing exhaustive information about the targeted application, the pros, and cons. Moreover, we present the characteristics of the human visual system presented in these models. In addition, we analyze and compare the 3D-JND models thoroughly using qualitative and quantitative performance evaluation based on Middlebury stereo datasets. Besides, we measure the JND thresholds of the asymmetric distortion based on psychophysical experiments and compare these experimental results to the estimates from the 3D-JND models in order to evaluate the accuracy of each model. Copyright the Author(s). Open Access under a CC BY 4.0 license. Users are allowed to read, download, copy, distribute, print, search, or link to the full texts of the articles, or use them for any other lawful purpose, as long as proper attribution is given.

One of the most challenging issues in stereoscopic image quality assessment (SIQA) is how to effectively model the binocular behavior of the human visual system (HVS). The latter has a great impact on the perceptual 3D quality. In this paper, we propose a SIQA metric accounting for binocular combination properties and disparity information. Instead of computing the quality of the left and the right views separately, the proposed metric predicts the quality of a cyclopean image so as to have a good consistency with 3D human perception. The cyclopean image is synthesized based on the local entropy and the visual saliency of each view with the aim to simulate the phenomena of binocular fusion/rivalry. A 2D IQA metric is employed to assess the quality of both the cyclopean image and the disparity map. The obtained scores are used to derive the 3D quality score thanks to a pooling stage. Experimental results on three public 3D IQA databases show that the proposed method outperforms many other state-of-the-art SIQA methods, and achieves high prediction accuracy on these databases.

One of the most challenging issues in stereoscopic image quality assessment (IQA) is how to effectively model the binocular behaviors of the human visual system (HVS). The latter has a great impact on the perceptual stereoscopic 3D (S3D) quality. This paper presents a stereoscopic IQA metric based on the properties of the HVS. Instead of measuring the quality of the left and the right views separately, the proposed method predicts the quality of a cyclopean image to ensure that the overall S3D quality is as close as possible to the binocular vision. The cyclopean image is synthesized based on the local entropy of each view with the aim to simulate the phenomena of the binocular rivalry/suppression. A 2D IQA metric is employed to assess the quality of both the cyclopean image and the disparity map. Additionally, the quality of the cyclopean image is modulated according to the visual importance of each pixel defined by the just noticeable difference (JND). Finally, the 3D quality score is derived by combining the quality estimates of the cyclopean image and disparity map. Experimental results show that the proposed method outperforms many other state-of-the-art SIQA methods in terms of prediction accuracy and computational efficiency.

International audience; The just noticeable difference (JND) notion reflects the maximum tolerable distortion. It has been extensively used for the optimization of 2D applications. For stereoscopic 3D (S3D) content, this notion is different since it relies on different mechanisms linked to our binocular vision. Unlike 2D, 3D-JND models appeared recently and the related literature is rather limited. These models can be used for the sake of compression and quality assessment improvement for S3D content. In this paper, we propose a deep and comparative study of the existing 3D-JND models. Additionally, in order to analyze their performance, the 3D-JND models have been integrated in recent metric dedicated to stereoscopic image quality assessment (SIQA). The results are reported on two widely used S3D image databases.

International audience; A video sequence is more than a sequence of still images. It contains a strong spatial-temporal correlation between the regions of consecutive frames. The most important characteristic of videos is the perceived motion foreground objects across the frames. The motion of foreground objects dramatically changes the importance of the objects in a scene and leads to a different saliency map of the frame representing the scene. This makes the saliency analysis of videos much more complicated than that of still images. In this paper, we investigate saliency in video sequences and propose a novel spatiotemporal saliency model devoted for video surveillance applications. Compared to classical saliency models based on still images, such as Itti's model, and space-time saliency models, the proposed model is more correlated to visual saliency perception of surveillance videos. Both bottom-up and top-down attention mechanisms are involved in this model. Stationary saliency and motion saliency are, respectively, analyzed. First, a new method for background subtraction and foreground extraction is developed based on content analysis of the scene in the domain of video surveillance. Then, a stationary saliency model is setup based on multiple features computed from the foreground. Every feature is analyzed with a multi-scale Gaussian pyramid, and all the features conspicuity maps are combined using different weights. The stationary model integrates faces as a supplement feature to other low level features such as color, intensity and orientation. Second, a motion saliency map is calculated using the statistics of the motion vectors field. Third, both motion saliency map and stationary saliency map are merged based on center-surround framework defined by an approximated Gaussian function. The video saliency maps computed from our model have been compared to the gaze maps obtained from subjective experiments with SMI eye tracker for surveillance video sequences. The results show strong correlation between the output of the proposed spatiotemporal saliency model and the experimental gaze maps.

The perception of video is different from that of image because of the motion information in video. Motion objects lead to the difference between two neighboring frames which is usually focused on. By far, most papers have contributed to image saliency but seldom to video saliency. Based on scene understanding, a new video saliency detection model with multi-features is proposed in this paper. First, background is extracted based on binary tree searching, then main features in the foreground is analyzed using a multi-scale perception model. The perception model integrates faces as a high level feature, as a supplement to other low-level features such as color, intensity and orientation. Motion saliency map is calculated using the statistic of the motion vector field. Finally, multi-feature conspicuities are merged with different weights. Compared with the gaze map from subjective experiments, the output of the multi-feature based video saliency detection model is close to gaze map.

The great advance of stereoscopic/3D technologies leads to a remarkable growth of 3D content in various applications thanks to a realistic and immersive experience. However, these technologies also brought some technical challenges and issues, regarding quality assessment and compression due to the complex processes of the binocular vision. Aiming to evaluate and optimize the performance of 3D imaging systems with respect to their storage capacity and quality of experience (QoE), this thesis focuses on two main parts: 1- spatial visibility thresholds of the human visual system (HVS) and 2- stereoscopic image quality assessment (SIQA). It is well-known that the HVS cannot detect the changes in a compressed image if these changes are lower than the just noticeable different (JND) threshold. Therefore, an extensive study based on objective and subjective analysis has been conducted on existing 3D-JND models. In addition, a new 3D-JND model has been proposed based on psychophysical experiments aiming to measure the effect of binocular disparity and spatial masking on the visual thresholds. In the second part, we explored new approaches for SIQA from two different perspectives. First, we developed a reference-based model accounting for both monocular and cyclopean quality. Then, we proposed a new blind quality metric relying on local contrast statistics combination of the stereopair. Both models considered the binocular fusion and binocular rivalry behaviors of the HVS in order to accurately simulate the human judgment of 3D quality.; La grande avancée des technologies stéréoscopiques/3D conduit à une croissance remarquable des contenus 3D dans diverses applications grâce aux effets immersifs qu’ils offrent. Cependant, ces technologies ont également créé de enjeux concernant l’évaluation de la qualité et la compression, en raison des processus complexes de la vision binocu laire. Visant à évaluer et à optimiser les performances des systèmes d’imagerie 3D quant à leur capacité de stockage et leur qualité d’expérience (QoE), cette thèse porte sur deux parties principales: 1- les seuils de visibilité spatiale du système visuel humain (SVH) et 2- l’évaluation de la qualité des images stéréoscopiques. Il est bien connu que le SVH ne peut détecter les modifications dans une image compressée si ces dernières sont inférieures au seuil JND (Just Noticeable Difference). Par conséquent, une étude approfondie basée sur une analyse objective et subjective a été menée sur les modèles 3D-JND existants. En outre, un nouveau modèle 3D-JND basé sur des expériences psychophysiques visant à mesurer l’effet de la disparité binoculaire et du masquage spatial sur les seuils visuels a été proposé. Dans la deuxième partie, nous avons exploré la mesure de la qualité 3D. Ainsi, nous avons développé un modèle avec référence prenant en compte à la fois la qualité monoculaire et cyclopéenne. Nous avons ensuite proposé une nouvelle métrique de qualité sans référence reposant sur la combinaison de statistiques locales de contraste de la paire stéréo. Les deux modèles reposent sur les propriétés de fusion et de rivalité binoculaire du SVH afin de simuler avec précision le jugement humain de la qualité 3D.

The great advance of stereoscopic/3D technologies leads to a remarkable growth of 3D content in various applications thanks to a realistic and immersive experience. However, these technologies also brought some technical challenges and issues, regarding quality assessment and compression due to the complex processes of the binocular vision. Aiming to evaluate and optimize the performance of 3D imaging systems with respect to their storage capacity and quality of experience (QoE), this thesis focuses on two main parts: 1- spatial visibility thresholds of the human visual system (HVS) and 2- stereoscopic image quality assessment (SIQA). It is well-known that the HVS cannot detect the changes in a compressed image if these changes are lower than the just noticeable different (JND) threshold. Therefore, an extensive study based on objective and subjective analysis has been conducted on existing 3D-JND models. In addition, a new 3D-JND model has been proposed based on psychophysical experiments aiming to measure the effect of binocular disparity and spatial masking on the visual thresholds. In the second part, we explored new approaches for SIQA from two different perspectives. First, we developed a reference-based model accounting for both monocular and cyclopean quality. Then, we proposed a new blind quality metric relying on local contrast statistics combination of the stereopair. Both models considered the binocular fusion and binocular rivalry behaviors of the HVS in order to accurately simulate the human judgment of 3D quality., La grande avancée des technologies stéréoscopiques/3D conduit à une croissance remarquable des contenus 3D dans diverses applications grâce aux effets immersifs qu’ils offrent. Cependant, ces technologies ont également créé de enjeux concernant l’évaluation de la qualité et la compression, en raison des processus complexes de la vision binocu laire. Visant à évaluer et à optimiser les performances des systèmes d’imagerie 3D quant à leur capacité de stockage et leur qualité d’expérience (QoE), cette thèse porte sur deux parties principales: 1- les seuils de visibilité spatiale du système visuel humain (SVH) et 2- l’évaluation de la qualité des images stéréoscopiques. Il est bien connu que le SVH ne peut détecter les modifications dans une image compressée si ces dernières sont inférieures au seuil JND (Just Noticeable Difference). Par conséquent, une étude approfondie basée sur une analyse objective et subjective a été menée sur les modèles 3D-JND existants. En outre, un nouveau modèle 3D-JND basé sur des expériences psychophysiques visant à mesurer l’effet de la disparité binoculaire et du masquage spatial sur les seuils visuels a été proposé. Dans la deuxième partie, nous avons exploré la mesure de la qualité 3D. Ainsi, nous avons développé un modèle avec référence prenant en compte à la fois la qualité monoculaire et cyclopéenne. Nous avons ensuite proposé une nouvelle métrique de qualité sans référence reposant sur la combinaison de statistiques locales de contraste de la paire stéréo. Les deux modèles reposent sur les propriétés de fusion et de rivalité binoculaire du SVH afin de simuler avec précision le jugement humain de la qualité 3D.