Search

International audience; Tone mapping operators (TMO) are functions which map high dynamic range (HDR) images to limited dynamic media while aiming to preserve the perceptual cues of the scene that govern its aesthetic quality. Evaluating aesthetic quality of TMOs is non-trivial due to the high subjectivity of preference involved. Traditionally, TMO aesthetic quality has been evaluated via subjective experiments in a controlled laboratory environment. However, the last decade has brought a surge in popularity of crowdsourcing as an alternative methodology to conduct subjective experiments. However, uncontrolled experiment conditions and unreliability of participant behaviour puts doubts on the trustworthiness of the collected data. In this study, we explore the possibility of using crowdsourcing platforms for subjective quality evaluation of TMOs. We have conducted three experiments with systematic changes to investigate the effect of experiment conditions and participant recruitment methods on the collected subjective data. Our results show that subjective evaluation of TMO aesthetic quality can be conducted on Prolific crowdsourcing platform with negligible differences in comparison to laboratory experiments. Furthermore, we provide objective conclusions about the effect of number of observers on the certainty of the pairwise comparison results.

International audience; Many objective quality metrics have been proposed over the years to automate the task of subjective quality assessment. However, few of them are designed for 3D graphical contents with appearance attributes; existing ones are based on geometry and color measures, yet they ignore the visual saliency of the objects. In this paper, we combined an optimal subset of geometry-based and color-based features, provided by a state-of-the-art quality metric for 3D colored meshes, with a visual attention complexity feature adapted to 3D graphics. The performance of our proposed new metric is evaluated on a dataset of 80 meshes with diffuse colors, generated from 5 source models corrupted by commonly used geometry and color distortions. With our proposed metric, we showed that the use of the attentional complexity feature brings a significant gain in performance and better stability.

International audience; In this paper, we propose a novel end-to-end system to extract strokes from offline math expressions. Using a multi-task neural network we simultaneously predict the location of the pen and the pen state. Our approach is based on a recent state-of-the-art image-to-sequence method limited to small fixed-sizes images. We generalize it to large and multi-symbol images without preprocessing steps such as skeletonization or binarization. This architecture allows an end-to-end training. A curriculum learning strategy have been used to address the complexity of the images. We achieve comparable results to the state of the art on the UNIPEN English character dataset considering the next point prediction. We propose a stroke level metrics that allows us to measure the stroke reconstruction. Experiments show the advantages and limitations of the adopted Image-to-Sequence method when scaling up to large and complex images such as math equations.

Nowadays, 360�� video/image has been increasingly popular and drawn great attention. The spherical viewing range of 360�� video/image accounts for huge data, which pose the challenges to 360�� video/image processing in solving the bottleneck of storage, transmission, etc. Accordingly, the recent years have witnessed the explosive emergence of works on 360�� video/image processing. In this paper, we review the state-of-the-art works on 360�� video/image processing from the aspects of perception, assessment and compression. First, this paper reviews both datasets and visual attention modelling approaches for 360�� video/image. Second, we survey the related works on both subjective and objective visual quality assessment (VQA) of 360�� video/image. Third, we overview the compression approaches for 360�� video/image, which either utilize the spherical characteristics or visual attention models. Finally, we summarize this overview paper and outlook the future research trends on 360�� video/image processing., Submitted to IEEE J-STSP SI of Perception-driven 360-degree video processing as an Invited Overview Paper

Objective: We describe a clinico-radiological presentation of inflammatory optic neuropathy that mimicked optic neuritis. Methods: Retrospective single-center case series and literature review of optic neuropathy without orbital pseudotumor. Results: Five local patients fulfilled the inclusion criteria. Clinical presentation revealed rapidly progressive severe unilateral visual loss, retrobulbar pain (n = 4), and paralytic strabismus (simultaneous = 2, protracted = 2) without proptosis. Optic nerve abnormality was not appreciated on initial scan review. Patients did not have any general activity of the granulomatosis with polyangiitis. Upon follow-up magnetic resonance imaging and initial imaging review, all patients revealed orbital apex anomalies. Visual acuity improved in three patients who received high-dose intravenous glucocorticosteroids immediately. Relapse was frequent and visual outcome was poor (final vision > 20/40 in two patients only). Literature review identified 16 well-documented cases of granulomatosis with polyangiitis–related isolated optic neuropathies. Magnetic resonance imaging revealed no abnormality (n = 6), optic nerve and/or sheath involvement (n = 9), apex infiltration (n = 3), and/or pachymeningitis (n = 7). Conclusion: Granulomatosis with polyangiitis is a rare yet potentially blinding cause of inflammatory optic neuropathy. Optic neuropathy in granulomatosis with polyangiitis may occur in the absence of systemic symptoms of disease activity and is challenging to distinguish from other inflammatory and non-inflammatory disorders affecting visual acuity. Several clinical and imaging clues suggest that optic neuropathy results from the development of an extravascular granulomatous process within the optic nerve sheath in the orbital apex, a place that is difficult to image. In a granulomatosis with polyangiitis patient with unexplained visual loss and a seemingly normal workup (fundoscopy, biology, and imaging), clinician should keep a high index of suspicion.

Sonar imagery plays a significant role in oceanic applications since there is little natural light underwater, and light is irrelevant to sonar imaging. Sonar images are very likely to be affected by various distortions during the process of transmission via the underwater acoustic channel for further analysis. At the receiving end, the reference image is unavailable due to the complex and changing underwater environment and our unfamiliarity with it. To the best of our knowledge, one of the important usages of sonar images is target recognition on the basis of contour information. The contour degradation degree for a sonar image is relevant to the distortions contained in it. To this end, we developed a new no-reference contour degradation measurement for perceiving the quality of sonar images. The sparsities of a series of transform coefficient matrices, which are descriptive of contour information, are first extracted as features from the frequency and spatial domains. The contour degradation degree for a sonar image is then measured by calculating the ratios of extracted features before and after filtering this sonar image. Finally, a bootstrap aggregating (bagging)-based support vector regression module is learned to capture the relationship between the contour degradation degree and the sonar image quality. The results of experiments validate that the proposed metric is competitive with the state-of-the-art reference-based quality metrics and outperforms the latest reference-free competitors.

Free-viewpoint video, as the development direction of the next-generation video technologies, uses the depth-image-based rendering (DIBR) technique for the synthesis of video sequences at viewpoints, where real captured videos are missing. As reference videos at multiple viewpoints are not available, a blind reliable real-time quality metric of the synthesized video is needed. Although no-reference quality metrics dedicated to synthesized views successfully evaluate synthesized images, they are not that effective when evaluating synthesized video due to additional temporal flicker distortion typical only for video. In this paper, a new fast no-reference quality metric of synthesized video with synthesis distortions is proposed. It is guided by the fact that the DIBR-synthesized images are characterized by increased high frequency content. The metric is designed under the assumption that the perceived quality of DIBR-synthesized video can be estimated by quantifying the selected areas in the high-high wavelet subband. The threshold is used to select the most important distortion sensitive regions. The proposed No-Reference Morphological Wavelet with Threshold (NR_MWT) metric is computationally extremely efficient, comparable to PSNR, as the morphological wavelet transformation uses very short filters and only integer arithmetic. It is completely blind, without using machine learning techniques. Tested on the publicly available dataset of synthesized video sequences characterized by synthesis distortions, the metric achieves better performances and higher computational efficiency than the state-of-the-art metrics dedicated to DIBR-synthesized images and videos.

Free-Viewpoint Video (FVV) systems allow the viewers to freely change the viewpoints of the scene. In such systems, view synthesis and compression are the two main sources of artifacts influencing the perceived quality. To assess this influence, quality evaluation studies are often carried out using conventional displays and generating predefined navigation trajectories mimicking the possible movement of the viewers when exploring the content. Nevertheless, as different trajectories may lead to different conclusions in terms of visual quality when benchmarking the performance of the systems, methods to identify critical trajectories are needed. This paper aims at exploring the impact of exploration trajectories (defined as Hypothetical Rendering Trajectories: HRT) on perceived quality of FVV subjectively and objectively, providing two main contributions. Firstly, a subjective assessment test including different HRTs was carried out and analyzed. The results demonstrate and quantify the influence of HRT in the perceived quality. Secondly, we propose a new objective video quality assessment measure to objectively predict the impact of HRT. This measure, based on Sketch-Token representation, models how the categories of the contours change spatially and temporally from a higher semantic level. Performance in comparison with existing quality metrics for FVV, highlight promising results for automatic detection of most critical HRTs for the benchmark of immersive systems., 11 pages, 7 figures

International audience; The last decade has brought a surge in crowdsourcing platforms' popularity for the subjective quality evaluation of multimedia content. The lower need for intervention during the experiment and more expansive participant pools of crowdsourcing platforms encourage researchers to join this trend. However, the unreliability of the participant behaviors puts a barrier in the wide adoption of these platforms. Although many works exist to detect unreliable observers in rating experiments, there is still a lack of methodology for detecting unreliable observers in quality evaluation of multimedia content using pairwise comparison. In this work, we propose methods to identify irregular annotator behaviors in pairwise comparison paradigm. We compare the proposed methods' efficiency for two scenarios: quality evaluation of traditional 2D images and 3D interactive multimedia. We conducted two crowdsourcing experiments for two different Quality of Experience assessment tasks and inserted carefully designed synthetic spammer profiles to evaluate the proposed tools. Our results suggest that the detection of unreliable observers is highly task-dependent. The influence of the spammer behavior intensity and the proportion of spammers among the observers can be more severe on tasks with higher subjectivity. Based on these findings, we provide guidelines and recommendations towards developing spammer detection algorithms for subjective pairwise quality evaluation of multimedia content.

International audience; Attention is an important attribute of human vision for study of user's quality of experience (QoE). The attention information collection from eye tracking is impossible in the current scenario of Covid-19. Different mouse metaphors have been proposed to study visual attention without eye tracking equipment. These methods have shown promising results on different types of images (visualizations, natural images and websites) with well-identified regions of interest. However, they have not been precisely tested for QoE applications, where natural images are processed with different algorithms (compression, tone-mapping, etc.) and visual content can induce more exploratory behavior. This paper studies and compares different configurations of bubble view metaphors for the study of visual attention in tone-mapped images.

International audience; A challenge for cognitive research is the better understanding of how motor activity influences and is influenced by other cognitive domains. We developed a preliminary study to investigate whether tracking motor functioning in virtual reality provides useful insight on cognitive functioning. We chose the flankers task as an assessment measure and built a VR environment into which seven participants completed more than 1250 trials. In addition to classical results of the flankers task showing that incongruent stimuli induce slower responses than others., we also identified how individuals are able to correct their initially incorrect motor response. This indicator may shed new light into the functioning of cognitive control in the future. We discuss the potential offered by virtual reality technology for cognitive assessment through embodied considerations of cognition.

Background In neuropsychology, fully immersive virtual reality (VR) has been spotlighted as a promising tool. It is considered that VR not only overcomes the existing limitation of neuropsychological tests but is also appropriate for treating executive functions (EFs) within activities of daily living (ADL) due to its high ecological validity. While fully immersive VR offers new possibilities of neuropsychological tests, there are few studies that overview the intellectual landscape and academic trends in the research related to mainly targeted EFs with fully immersive VR. Objective The objective of this study is to get an overview of the research trends that use VR in neuropsychological tests and to analyze the research trends using fully immersive VR neuropsychological tests with experimental articles. Methods This review was carried out according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. Articles were searched in three web databases using keywords related to VR, EFs, and cognitive abilities. The study was conducted in two steps, keyword analysis and in-depth systematic review. In the web database search from 2000 to 2019, 1167 articles were initially collected, of which 234 articles in the eligibility phase were used to conduct keyword analysis and a total of 47 articles were included for systematic review. Results In keyword analysis, the number of articles focused on dementia including the keywords “MCI,” “SCD,” and “dementia” were highlighted over the period, rather than other symptoms. In addition, we identified that the use of behavioral and physiological data in virtual environments (VEs) has dramatically increased in recent studies. In the systematic review, we focused on the purpose of study, assessment, treatment, and validation of usability and structure. We found that treatment studies and uncategorized studies including presence and cybersickness issues have emerged in the recent period. In addition, the target symptoms and range of participants were diversified. Conclusions There has been a continuously increasing interest in dealing with neuropsychology by using fully immersive VR. Target cognitive abilities have been diversified, as well as target symptoms. Moreover, the concept of embodied cognition was transplanted in this research area.

Objective image quality metrics try to estimate the perceptual quality of the given image by considering the characteristics of the human visual system. However, it is possible that the metrics produce different quality scores even for two images that are perceptually indistinguishable by human viewers, which have not been considered in the existing studies related to objective quality assessment. In this paper, we address the issue of ambiguity of objective image quality assessment. We propose an approach to obtain an ambiguity interval of an objective metric, within which the quality score difference is not perceptually significant. In particular, we use the visual difference predictor, which can consider viewing conditions that are important for visual quality perception. In order to demonstrate the usefulness of the proposed approach, we conduct experiments with 33 state-of-the-art image quality metrics in the viewpoint of their accuracy and ambiguity for three image quality databases. The results show that the ambiguity intervals can be applied as an additional figure of merit when conventional performance measurement does not determine superiority between the metrics. The effect of the viewing distance on the ambiguity interval is also shown.

International audience

National audience

International audience; Recent advances in eXplainable Artificial Intelligence (XAI) led to many different methods in order to improve explainability of deep learning algorithms. With many options at hand, and maybe the need to adapt existing ones to new problems, one may find in a struggle to choose the right method to generate explanations. This paper presents an objective approach to compare two different existing XAI methods. These methods are applied to a use case from literature and to a real use case of a French administration.

International audience; In this paper, we propose a novel framework to characterize a wide color gamut image content based on perceived quality due to the processes that change color gamut, and demonstrate two practical use cases where the framework can be applied. We first introduce the main framework and implementation details. Then, we provide analysis for understanding of existing wide color gamut datasets with quantitative characterization criteria on their characteristics, where four criteria, i.e., coverage, total coverage, uniformity, and total uniformity, are proposed. Finally, the framework is applied to content selection in a gamut mapping evaluation scenario in order to enhance reliability and robustness of the evaluation results. As a result, the framework fulfils content characterization for studies where quality of experience of wide color gamut stimuli is involved.

International audience; Tone-mapping operator (TMO) plays a crucial role in the task towards displaying high dynamic range (HDR) contents on standard displays. Similarly, visual attention (VA) is a vital feature of the human visual system (HVS), while it has not been sufficiently investigated in preference of tone-mapped images. The potential benefits of visual attention-based features for quality assessment of tone-mapped images are studied in this paper. A novel framework is proposed for tone-mapped image quality assessment to predict image preference. The framework is evaluated on two different datasets. Experimental results illustrate the importance of visual attention for improving the performance of objective metrics. The proposed framework outperforms the existing methods and presents as a competitive alternative for tone-mapped images evaluation.

International audience; Understanding human visual attention mechanisms and interaction in immersive scenes are of great importance in perception. In immersive context, users are able to interact with increasingly rich/ complex 3D contents during rendering. Therefore, to avoid latency or rendering issues, there is a critical need for simplifying and filtering the primitives and levels of detail of these high-quality 3D graphics (according to viewing conditions). In order to ensure a high user's quality of experience (QoE) during interactive visualization, these processing operations should take into account perceptual information. To do so, we suggest an approach that uses visual saliency information of the 3D scene to guide simplification and level of details selection.In this paper, we question the efficiency of our novel approach to compute visual saliency on 3D graphics. This approach takes into consideration the viewpoint from which the 3D content was seen/rendered when computing saliency (whichever the considered viewpoint is), by using saliency maps of view-based method computed offline. Such technique could help alleviate rendering constraints during interactive visualization.

This paper provides insights on how to perceptually characterize colored 3D Graphical Contents (3DGC). In this study, pre-defined viewpoints were considered to render static graphical objects. For perceptual characterization, we used visual attention complexity (VAC) measures. Considering a view-based approach to exploit the perceived information, an eye-tracking experiment was conducted using colored graphical objects. Based on the collected gaze data, we revised the VAC measure, suggested in 2D imaging context, and adapted it to 3DGC. We also provided an objective predictor that highly mimics the experimental attentional complexity information. This predictor can be useful in Quality of Experience (QoE) studies: to balance content selection when benchmarking 3DGC processing techniques (e.g., rendering, coding, streaming, etc.) for human panel studies or ad hoc key performance indicator, and also to optimize the user's QoE when rendering such contents.

International audience; At lower bit-rate encoding video in real-time with a reasonable viewing quality is challenging. Content adaptive per-title encoding is usually leveraged for OTT/VOD delivery by selecting the optimal resolutions and qualities of a given video using multiple encodings. Built on such powerful resolution selection principles, this paper introduces an on the fly resolution prediction without requiring multiple encoding with the help of machine learning which is suitable for real-time video delivery. Two machine learning networks are defined based on the resolution of the previous decision period. Three types of machine learning classifiers: weighted SVM, Random Forests (RF), and custom-designed Multi-Layer Perceptron (MLP) are tested. Suitability of classifiers for real-time resolution prediction is discussed based on the accuracy, BDrate performances, and impact of misclassification on encoding performance and hardware implementability. The proposed solution offers a promising average bit-rate savings upto 12.6%.

International audience; Recent advancements in the image capturing techniques and post processing software generate High Dynamic Range (HDR) images. Such images retain maximum information of the scene by capturing more realistic visual contents, which are often missed in traditional image capturing techniques. In this regard, tone mapping operators (TMOs) play a significant role in displaying HDR image contents on a traditional Low Dynamic Range (LDR) display. These operators tend to introduce artifacts in the original HDR image to change its brightness and contrast in such a way that it can destroy the important textures and information of the image. The assessment of these TMOs is a challenging topic to select best the best technique considering different perceptual and quality dimensions. In this paper, we propose to compare TMOs through their impact on visual behavior in comparison with HDR condition. This study is the first of its kind to utilize hidden markov model (HMM) as a similarity measure to evaluate perceived quality of TMO. The findings suggest that the proposed HMM-based method which emphasizes on temporal information produce better evaluation metric than the traditional approaches which are based only on visual spatial information.

In recent years, the rapid development of diverse media has been evident in disparate fields such as consumer electronics, automotive infotainment and healthcare software. There is a need for innovative methods to assess user perceived Quality of Experience (QoE), as a proxy for consumer satisfaction with such systems and services. Users emotional state plays a key role in QoE; thus, it is necessary to consider it in user experience evaluation and the design process of stereoscopic 3D video content. In the present article we introduce the use of a specially designed model based on a feedforward Multilayer Perception Artificial Neural Network as an appropriate Machine Learning technique for the estimation of human emotional state while viewing various categories of stereoscopic 3D video content. The goal is to design an emotional state estimator based on direct psychophysiological measurements. The considered psychophysiological signals include heart rate (HR) calculated from an echocardiogram (ECG), electro-dermal activity (EDA), and brain activity (BA) in EEG signals. Participants watched a series of 3D video contents varying in terms of visual quality, while the mentioned psychophysiological signals were recorded, and self-reported subjectively experienced emotions using a Self-Assessment Manikin (SAM) questionnaire. The obtained results show that it is possible to construct such a highly precise estimator of emotional states.

International audience; 18 FDG PET/CT imaging is commonly used in diagnosis and follow-up of metastatic breast cancer, but its quantitative analysis is complicated by the number and location heterogeneity of metastatic lesions. Considering that bones are the most common location among metastatic sites, this work aims to compare different approaches to segment the bones and bone metastatic lesions in breast cancer. Two deep learning methods based on U-Net were developed and trained to segment either both bones and bone lesions or bone lesions alone on PET/CT images. These methods were cross-validated on 24 patients from the prospective EPICURE seinmeta metastatic breast cancer study and were evaluated using recall and precision to measure lesion detection, as well as the Dice score to assess bones and bone lesions segmentation accuracy. Results show that taking into account bone information in the training process allows to improve the precision of the lesions detection as well as the Dice score of the segmented lesions. Moreover, using the obtained bone and bone lesion masks, we were able to compute a PET bone index (PBI) inspired by the recognized Bone Scan Index (BSI). This automatically computed PBI globally agrees with the one calculated from ground truth delineations. Clinical relevance - We propose a completely automatic deep learning based method to detect and segment bones and bone lesions on 18 FDG PET/CT in the context of metastatic breast cancer. We also introduce an automatic PET bone index which could be incorporated in the monitoring and decision process.

National audience; Dans le domaine de la fécondation in vitro (FIV), l'imagerie par time-lapse (ITL) est une technologie qui produit une vidéo montrant le développement de l'embryon pendant ses premiers jours. L'ITL permet d'annoter les instants auxquels commencent et finissent les différentes phases de développement de l'embryon. Cependant, le processus d'annotation prend du temps et nécessite des experts. Dans cet article, nous montrons que pour prédire les phases visibles dans les images issues de vidéos ITL, un ResNet-3D est meilleur qu'un équivalent 2D ou un ResNet-LSTM.

National audience

National audience

High-definition 360 videos encoded in fine quality are typically too large in size to stream in its entirety over bandwidth (BW)-constrained networks. One popular remedy is to interactively extract and send a spatial sub-region corresponding to a viewer’s current field-of-view (FoV) in a head-mounted display (HMD) for more BW-efficient streaming. Due to the non-negligible round-trip-time (RTT) delay between server and client, accurate head movement prediction that foretells a viewer’s future FoVs is essential. Existing approaches are either overly simplistic in modelling and predict poorly when RTT is large, or are over-reliant on data-driven learning, resulting in inflexible models that are not robust to RTT heterogeneity. In this paper, we cast the head movement prediction task as a sparse directed graph learning problem, where three sources of relevant information—a 360 image saliency map, collected viewers’ head movement traces, and a biological head rotation model—are aggregated into a unified Markov model. Specifically, we formulate a constrained optimization problem to minimize an l 2 -norm fidelity term and a sparsity term, corresponding to trace data / saliency consistency and a sparse graph model prior respectively. We solve the problem alternately using a hybrid iterative reweighted least square (IRLS) and Frank-Wolfe optimization strategy. Extensive experiments show that our head movement prediction scheme noticeably outperforms existing proposals across a wide range of RTTs.

With the development of immersive media technologies, omnidirectional video service has been popularized at an accelerated speed due to the sense of presence it offered. Accurately measuring the user’s subjective sense of presence on virtual reality (VR) devices are urgently required by video service providers and VR device manufacturers to further improve their quality of service. So far, there is no standardized methodology available for the conduction of the subjective experiment toward the user’s sense of presence when viewing omnidirectional videos. In this paper, we designed a novel subjective assessment method, which evaluates the presence of omnidirectional videos where the scenes have been previously experienced in the real world by subjects. In this setup, subjects can easily and accurately form their sense of presence when viewing omnidirectional videos using previous observations of the real environment as references. The experimental results demonstrated that the proposed method is able to obtain reliable quantitative results toward the sense of presence.

In this work, we develop a novel no-reference (NR) quality assessment metric for stereoscopic images based on monocular and binocular features, motivated by visual perception properties of the human visual system (HVS) named binocular rivalry and binocular integration. To be more specific, we first calculate the normalized intensity feature maps of right- and left-view images through local contrast normalization, where statistical intensity features are extracted by the histogram of the normalized intensity feature map to represent monocular features. Then, we compute the disparity map of stereoscopic image, with which we extract structure feature map of stereoscopic image based on local binary pattern (LBP). We further extract statistical structure features and statistical depth features from structure feature map and disparity map by histogram to represent binocular features. Finally, we adopt support vector regression (SVR) to train the mapping function from the extracted monocular and binocular features to subjective quality scores. Comparison experiments are conducted on four large-scale stereoscopic image databases and the results demonstrate the promising performance of the proposed method in stereoscopic image quality assessment.

Traditionally, the quality of a multimedia system was mainly assessed through the evaluation of its Quality of Service (QoS) that is by evaluating system parameters such as bandwidth, latency, jitter, throughput, transmission delay, availability, etc. However, these metrics often failed to capture the actual end-user perceived quality, which has prompted the development of the construct of Quality of Experience (QoE), widely understood as an interaction of the technical features of multimedia systems with perceptual, and cognitive/emotional factors involved in the interpretation of those features by users. This chapter addresses the open issues in the field of QoS and QoE assessments. First, the perceptual characteristics of the multiview content are analyzed, and then a survey on the existing approaches for QoS and QoE estimation is performed. The analysis is then focused on the subjective aspects of QoE assessment, by describing the standard methodologies currently used and new trends based on human factors research. Finally, the chapter offers a few guidelines for future research directions in the field.

In submarine and underwater detection tasks, conventional optical imaging and analysis methods are not universally applicable due to the limited penetration depth of visible light. Instead, sonar imaging has become a preferred alternative. However, the capture and transmission conditions in complicated and dynamic underwater environments inevitably lead to visual quality degradation of sonar images, which might also impede further recognition, analysis and understanding. To measure this quality decrease and provide a solid quality indicator for sonar image enhancement, we propose a task- and perception-oriented sonar image quality assessment ( TPSIQA ) method, in which a semi-reference (SR) approach is applied to adapt to the limited bandwidth of underwater communication channels. In particular, we exploit reduced visual features that are critical for both human perception of and object recognition in sonar images. The final quality indicator is obtained through ensemble learning, which aggregates an optimal subset of multiple base learners to achieve both high accuracy and a high generalization ability. In this way, we are able to develop a compact but generalized quality metric using a small database of sonar images. Experimental results demonstrate competitive performance, high efficiency, and strong robustness of our method compared to the latest available image quality metrics.

International audience

In image/video systems, contrast adjustment which manages to enhance visual quality is nowadays an important research topic. Yet very limited struggles have been made to the exploration of visual quality assessment for contrast adjustment. To tackle the issue, this paper proposes a novel reduced-reference (RR) quality metric with the integration of bottom-up and top-down strategies. The former one stems from the recently revealed free energy principle that tells that the human visual system seeks to comprehend an input image via uncertainty removal, while the latter one is toward using the symmetric Kullback–Leibler divergence to compare the histogram of the contrast-altered image with that of the pristine image. The bottom-up and top-down strategies are lastly incorporated to derive the RR contrast-altered image quality measure. A comparison using numerous existing IQA models is carried out on five contrast related databases/subsets in CID2013, CCID2014, CSIQ, TID2008, and TID2013, and experimental results validate the superiority of the proposed technique.

International audience; Wireless capsule endoscopy (WCE) allows medical doctors to examine the interior of the small intestine with a noninvasive procedure. This methodology is particularly important for Crohn's disease (CD), where an early diagnosis improves treatment outcomes. The counting and identification of CD lesions in WCE videos is a time-consuming process for medical experts. In the era of deep-learning many automatic WCE lesion classifiers, requiring annotated data, have been developed. However, benchmarking classifiers is difficult due to the lack of standard evaluation data. Most detection algorithms are evaluated on private datasets or on unspecified subsets of public databases. To help the development and comparison of automatic CD lesion classifiers, we release CrohnIPI, a dataset of 3498 images, independently reviewed by several experts. It contains 60.55% of non-pathological images and 38.85% of pathological images with 7 different types of CD lesions. A part of these images are multilabeled. The dataset is balanced between pathological images and non-pathological ones and split into two subsets for training and testing models. This database will progressively be enriched over the next few years in aim to make the automatic detection algorithms converge to the most accurate system possible and to consolidate their evaluation.

On ne peut que constater combien la technologie vidéo est devenue omniprésente en un peu plus d’une décennie. Si les besoins des nouveaux services vidéo ont été en partie satisfaits par l’efficacité des réseaux et de leurs technologies de couche physique, en parallèle les codecs vidéo ont été primordiaux en offrant compression et nouvelles fonctionnalités. La recherche en codage vidéo se caractérise alors par un fort contexte normatif où le point de compatibilité se fait au niveau du décodage. Les performances de chaque nouveau standard sont remarquables, car typiquement chacun par rapport au précédent, code avec la même qualité objective un format vidéo donné mais en divisant par deux le débit nécessaire. Il est cependant frappant de constater que le paradigme de la couche de codage n’a pas changé depuis l’origine, avec un codec dissymétrique où les décisions sont prises à l’encodage via un schéma hybride intégrant les étapes de prédiction, transformation, quantification et codage entropique. Il est aussi surprenant de constater la persistance de l’implantation dans les codeurs pour la phase d’optimisation débit-distorsion, d’un calcul de distorsion de type erreur quadratique moyenne, alors que l’on sait la corrélation pauvre entre les résultats de cette métrique et le jugement humain. C’est à ce niveau que se définit l’objectif du codage vidéo perceptuel qui est d’aller vers une représentation de l'information basée sur le contenu perçu et qui puisse servir de paradigme pour un codage basé qualité perceptuelle des contenus des vidéos 2D et 3D. Pour être en phase avec les approches standards, l’objectif devient de guider les choix de codage de façon à aller vers la fidélité perceptuelle sans perte des vidéos décodées. Mes travaux se positionnent donc dans cette recherche des outils et des méthodes où les choix pour améliorer la qualité perceptuelle guident le codage vidéo.Mon mémoire retrace mon itinéraire scientifique au sein des équipes IVC IRCCyN puis IPI LS2N qui, du fait des recherche qui y sont conduites sur la modélisation du système visuel humain, ont offert et offrent encore un cadre collaboratif particulièrement propice aux travaux liés au codage vidéo perceptuel. Mes contributions au codage vidéo perceptuel se sont principalement organisées au sein de projets de recherche visant à contribuer aux efforts faits en normalisation, avec le projet ANR ArchiPEG qui avait pour objectif l'optimisation du codage vidéo HD en flux H.264/AVC, le projet ANR PERSEE qui s'intéressait au codage perceptuel des formats vidéo 2D et 3D via le codec H.264 et son extension MVC, et le projet européen ITN PROVISION qui visait le codage perceptuel via HEVC. Mes contributions ont été produites au travers l’encadrement de masters et de thèses (4 soutenues, 1 en cours), et nous verrons que nous avons agi à 2 niveaux de la boucle de codage vidéo hybride.Une première possibilité est d’améliorer directement des unités de la boucle de codage. C’est ainsi que nous avons mis en œuvre une transformation directionnelle tenant compte des orientations géométriques au sein de l’image. Nous avons exploité cette représentation de l’information afin de construire des classes de structures, puis adapté la quantification de leur contenu en se basant sur des critères perceptuels. Nous avons aussi, après avoir défini la notion de textures spatio-temporelles visuelles, proposé un modèle mesurant la similarité entre ces textures visuelles. Ce modèle computationnel basé sur le traitement cortical du SVH, a ensuite été utilisé afin d’améliorer la boucle de codage de HEVC : en identifiant des classes de textures visuelles et en faisant correspondre à la distorsion objective standard calculée par le codeur sa valeur dans le domaine perceptuel. Une autre approche consiste à mettre en place un schéma de pré-analyse avant encodage afin de piloter ensuite efficacement le codage des régions d’intérêt. La modélisation de l’attention visuelle est donc essentielle au niveau de cette analyse. Nous avons conçu un schéma de pré-analyse basé sur des tubes spatio-temporels. Cet outil réalise la segmentation et le suivi des objets constitués de tubes connexes, et il leur associe une saillance visuelle. L’ensemble de ces informations est exploitée afin d’assurer un codage cohérent des objets en flux H.264/AVC. Exactement nous avons testé l’adaptation de la structure du GOP, et le guidage de la qualité visuelle. Nous avons aussi développé le premier modèle computationnel de l’attention visuelle 3D s’appliquant à la visualisation d’images stéréoscopiques. Ce modèle combine deux cartes de saillance visuelle : celle 2D et celle de profondeur. Nous avons utilisé ce modèle afin d’améliorer la qualité d’expérience de visualisation de la TV3D, par ajout d’un flou de défocalisation sur la région ayant moins d’intérêt. Dans la dernière partie de mon mémoire, un bilan et des perspectives sont donnés. Nous nous intéressons en particulier au codage des formats vidéo immersifs représentés par des nuages de points 3D. Nos derniers travaux montrent l’intérêt d’une approche basée quantification vectorielle algébrique et arborescente, permettant de structurer graduellement la géométrie de la scène, et cela en fonction d’un critère débit-distorsion.

Owning to the recorded light ray distributions, light field contains much richer information and provides possibilities of some enlightening applications, and it has becoming more and more popular. To facilitate the relevant applications, many light field processing techniques have been proposed recently. These operations also bring the loss of visual quality, and thus there is need of a light field quality metric to quantify the visual quality loss. To reduce the processing complexity and resource consumption, light fields are generally sparsely sampled, compressed, and finally reconstructed and displayed to the users. We consider the distortions introduced in this typical light field processing chain, and propose a full-reference light field quality metric. Specifically, we measure the light field quality from three aspects: global spatial quality based on view structure matching, local spatial quality based on near-edge mean square error, and angular quality based on multi-view quality analysis. These three aspects have captured the most common distortions introduced in light field processing, including global distortions like blur and blocking, local geometric distortions like ghosting and stretching, and angular distortions like flickering and sampling. Experimental results show that the proposed method can estimate light field quality accurately, and it outperforms the state-of-the-art quality metrics which may be effective for light field.

International audience; Visual attention has been shown as a good proxy for QoE, revealing specific visual patterns considering content, system and contextual aspects of a multimedia applications. In this paper, we propose a novel approach based on hidden markov models to analyze visual scanpaths in an image memorability task. This new method ensures the consideration of both temporal and idiosyncrasic aspects of visual behavior. The study shows promising results for the use of indirect measures for the personalization of QoE assessment and prediction.

International audience; As an emerging technology, Light Field (LF) has gained ever-increasing importance in the domain. In order to provide guidance for the development of perceptually accurate Light Field (LF) processing tools and supervise the entire streaming system, robust perceptual quality assessment metrics are required. Especially, No-Reference (NR) metrics are preferable to compare LF with different angular resolutions. Some metrics have been developed by extending commonly used 2D image quality metrics to the 4D LF domain with angular consistency terms. Nonetheless, although these models consistently show slightly improved performance, most of them are limited in evaluating the quality of LF using the sub-aperture views with additional terms on the angular domain. There is an evident lack of reliable quality metrics that are tailored to LF content. To remedy this lack, we propose a NR quality metric for LF contents based on representing EPI with structural descriptors, including the Histogram of Gradients and the Convolutional Sparse Coding based de-scriptors. The primary motivation resides in our observations that (1) LF related distortions on the angular domain are highly noticeable on the Epipolar Plane Image representations (EPI); (2) most of the distortions in EPI are structure-related. Extensive experiments on the MPI-LFA [1] LF image quality dataset demonstrate that our method provides competitive performance with the state-of-the-art NR image quality metrics.

International audience; Virtual reality (VR) headsets offer a large and immersive workspace for displaying visualizations with stereo-scopic vision, compared to traditional environments with monitors or printouts. The controllers for these devices further allow direct three-dimensional interaction with the virtual environment. In this paper, we make use of these advantages to implement a novel multiple and coordinated view (MCV) in the form of a vertical stack, showing tilted layers of geospatial data to facilitate an understanding of multi-layered maps. A formal study based on a use-case from urbanism that requires cross-referencing four layers of geospatial urban data augments our arguments for it by comparing it to more conventional systems similarly implemented in VR: a simpler grid of layers, and switching (blitting) layers on one map. Performance and oculometric analyses showed an advantage of the two spatial-multiplexing methods (the grid or the stack) over the temporal mul-tiplexing in blitting. Overall, users tended to prefer the stack, be ambivalent to the grid, and show dislike for the blitting map. Perhaps more interestingly, we were also able to associate preferences in systems with user characteristics and behavior.

International audience; Tone mapping operators (TMO) are pivotal in rendering High Dynamic Range (HDR) content on limited dynamic range media. Analysing the quality of tone mapped images depends on several objective factors and a combination of several subjective factors like aesthetics, fidelity etc. Objective Image quality assessment (IQA) metrics are often used to evaluate TMO quality but they do not always reflect the ground truth. A robust alternative to objective IQA metrics is subjective quality assessment. Although, subjective experiments provide accurate results, they can be time-consuming and expensive to conduct. Over the last decade, crowdsourcing experiments have become more popular for collecting large amount of data within a shorter period of time for a lesser cost. Although they provide more data requiring less resources, lack of controlled environment for the experiment results in noisy data. In this work 1 , we propose a comprehensive analysis of crowdsourcing experiments with two different groups of participants. Our contributions include a comparative study and a collection of methods to detect unreliable participants in crowdsourcing experiments in a TMO quality evaluation scenario. These methods can be utilized by the scientific community to increase the reliability of the gathered data.

User expectations have a crucial impact on the levels of quality of experience (QoE) that they consider acceptable or satisfying. Measuring acceptability and annoyance has mainly been performed in separate or multi-step experiments without any control over participants’ expectations. This paper introduces a simple methodology to obtain the information about both of the entities in a single step and compares several data processing strategies useful for results interpretation. A specifically designed subjective experiment, conducted on compressed videos, has shown that the multi-step procedures could be replaced by our proposed single-step approach, regardless of the viewing conditions, while the novel approach is significantly preferred by observers for its low time requirements and higher intuitiveness. The test has simultaneously proven that user expectations can be altered by the instructions and it is, therefore, possible to simulate different user profiles regardless of the participants’ real habits. The acceptability/annoyance experimental results are also used to benchmark the state-of-the-art objective video quality metrics in predicting acceptability/annoyance of QoE. A case study on the determination of the threshold of acceptability/annoyance for objective quality metrics is conducted, which can be served as a guideline for video streaming service providers.

International audience; Visual attention is a key feature to optimize visual experience of many multimedia applications. 2D visual attention computational modeling is an active research area considering the visualization of natural images on a conventional display. In this paper, we question the ability of such models to be applicable to single computer-generated objects rendered at different sizes (on a conventional display). We benchmark state of art visual attention models and investigate the influence of the viewpoint on those computational models applied on volumetric data and this to get a better understanding of how viewpoint information could be integrated into view-based approaches. To do so, a subjective experiment was conducted and a fine analysis was led using the variance analysis statistical method.

International audience; Wireless capsule endoscopy (WCE) allows medical doctors to examine the interior of the small intestine with a non-invasive procedure. This methodology is particularly important for Crohn's disease (CD), where an early diagnosis improves treatment outcomes. However, the viewing and evaluation of WCE videos is a time-consuming process for the medical experts. In this work, we present a recurrent attention neural network for the detection in WCE images of CD lesions in the small bowel. Our classifier reaches 90.85% accuracy on our own dataset annotated by experts from the Hospital of Nantes. The model has also been tested on a public endoscopic dataset, the CAD-CAP database used for the GIANA competition, and achieves high performance on detection task with an accuracy of 99,67%. This automatic lesion classifier will greatly reduce the amount of time spent by gastroenterologists in reviewing WCE videos, which will likely foster the development of this technique and speed-up the diagnosis of CD.

International audience; In order to predict where humans look in a 3D immersive environment, saliency can be computed using either 3D saliency models or view-based approaches (2D projection). In fact, building a 3D complete model is still a challenging task that is not investigated enough in the research field while 2D imaging approaches have been extensively studied and have shown solid performances. As 6 degrees of freedom are allowed in volumetric videos, users are able to navigate through the content in different manners. In this case, 2D saliency models might be less robust if applied naïvely, since advanced parameters such as viewing distance are not considered in such models. The aim of this paper is to investigate the influence of viewpoint on 2D saliency models when applied on volumetric data and this to get a better understanding of how viewpoint information could be integrated into view-based approaches. To do so, a subjective psycho-visual experiment was conducted and a fine analysis was led using the variance analysis statistical method.

International audience; The objective assessment of encoding performance is a key aspect of video delivery optimization. Objective metrics typically do not address fully the different viewing distances and behavior of compression artifacts being subjected to perceptual changes in video. This poses a daunting task of optimizing compression for video delivery systems for specific viewing conditions and perceptual optimization. This paper introduces a perceptual pre-processing and then discusses accuracy of typically used objective metrics for judging performance of pre-processing for observers at different viewing distances. To this end, this paper reports an in-depth analysis to check if objective metrics can successfully match subjective results at critical pairs i.e. pre-processed and original video at same QP.

International audience; We summarize the tasks, protocol, and outcome for the 6th Competition on Recognition of Handwritten Mathematical Expressions (CROHME), which includes a new formula detection in document images task (+ TFD). For CROHME + TFD 2019, participants chose between two tasks for recognizing handwritten formulas from 1) online stroke data, or 2) images generated from the handwritten strokes. To compare L A T E X strings and the labeled directed trees over strokes (label graphs) used in previous CROHMEs, we convert LATEX and stroke-based label graphs to label graphs defined over symbols (symbol-level label graphs, or symLG). More than thirty (33) participants registered for the competition, with nineteen (19) teams submitting results. The strongest formula recognition results were produced by the USTC-iFLYTEK research team, for both stroke-based (81%) and image-based (77%) input. For the new typeset formula detection task, the Samsung R&D Institute Ukraine (Team 2) obtained a very strong F-score (93%). System performance has improved since the last CROHME-still, the competition results suggest that recognition of handwritten formulae remains a difficult structural pattern recognition task.

International audience; This paper proposes a perceptually optimized pre-processing technique using state of the art Human Visual System (HVS) model suitable for video compression to reduce bit-rate at the same quality. Visual masking models to accurately account HVS has been considered. Frequencies which are visually indistinguishable and need not be encoded are removed within visibility threshold. The scheme is optimized for multiple viewing distances to consider real-world scenarios. Extensive subjective and objective evaluation has been conducted to evaluate proposed pre-processing. Investigation shows significant bit-rate savings compared to a professional real time HEVC video encoder.

As the immersive multimedia techniques like Free-viewpoint TV (FTV) develop at an astonishing rate, user's demand for high-quality immersive contents increases dramatically. Unlike traditional uniform artifacts, the distortions within immersive contents could be non-uniform structure-related and thus are challenging for commonly used quality metrics. Recent studies have demonstrated that the representation of visual features can be extracted from multiple levels of the hierarchy. Inspired by the hierarchical representation mechanism in the human visual system (HVS), in this paper, we explore to adopt structural representations to quantitatively measure the impact of such structure-related distortion on perceived quality in FTV scenario. More specifically, a bio-inspired full reference image quality metric is proposed based on 1) low-level contour descriptor; 2) mid-level contour category descriptor; and 3) task-oriented non-natural structure descriptor. The experimental results show that the proposed model outperforms significantly the state-of-the-art metrics.