Search

This work presents a pose estimation method from a RGB image of industrial parts placed in a bin. In a first time, neural networks are used to segment a certain number of parts in the scene. After applying an object mask to the original image, a second network is inferring the local depth of the part. Both the local pixel coordinates of the part and the local depth are used in two networks estimating the orientation of the object as a quaternion and its translation on the Z axis. Finally, a registration module working on the back-projected local depth and the 3D model of the part is refining the pose inferred from the previous networks. To deal with the lack of annotated real images in an industrial context, an data generation process is proposed. By using various light parameters, the dataset versatility allows to anticipate multiple challenging exploitation scenarios within an industrial environment.; Ces travaux de thèse présentent une méthode d’estimation de pose de pièces industrielles en vue de leur dévracage à partir d’un système mono-caméra 2D en utilisant une approche par apprentissage avec des réseaux profonds. Dans un premier temps, des réseaux de neurones assurent la segmentation d’un nombre prédéterminé de pièces dans la scène. En appliquant le masque binaire d’une pièce à l’image originale, un second réseau infère la profondeur locale de cet objet. En parallèle des coordonnées de la pièce dans l’image, cette profondeur est employée dans deux réseaux estimant à la fois l’orientation de l’objet sous la forme d’un quaternion et sa translation sur l’axe Z. Enfin, un module de recalage travaillant sur la rétro-projection de la profondeur et le modèle 3D de l’objet, permet d’affiner la pose prédite par les réseaux. Afin de pallier le manque de données réelles annotées dans un contexte industriel, un processus de création de données synthétiques est proposé. En effectuant des rendus aux multiples luminosités, la versatilité du jeu de données permet d’anticiper les différentes conditions hostiles d’exploitation du réseau dans un environnement de production.

Development and performance evaluation of efficient methods for coding, transmission, and quality assessment of 3D visual content require rich datasets of a suitable test material. The use of these databases allows a fair comparison of systems under test. Moreover, publicly available and widely used datasets are crucial for experimentation leading to reproducible research. This chapter presents an overview of 3D visual content datasets relevant to research in the field of coding, transmission, and quality assessment. Description of regular stereoscopic or multiview image and video datasets is presented. Databases created using emerging technologies, including light-field imaging, are also addressed. Moreover, there are databases of multimedia content annotated with ratings from the subjective experiment, which are a necessary resource for understanding the complex problem of quality of experience while consuming the 3D visual content.

International audience; In this paper, we address the challenging problem of the 3D point cloud compression required to ensure efficient transmission and storage. We introduce a new hierarchical geometry representation based on adaptive Tree-Structured Point-Lattice Vector Quantization (TSPLVQ). This representation enables hierarchically structured 3D content that improves the compression performance for static point cloud. The novelty of the proposed scheme lies in adaptive selection of the optimal quantization scheme of the geometric information, that better leverage the intrinsic correlations in point cloud. Based on its adaptive and multiscale structure, two quantization schemes are dedicated to project recursively the 3D point clouds into a series of embedded truncated cubic lattices. At each step of the process, the optimal quantization scheme is selected according to a rate-distortion cost in order to achieve the best trade-off between coding rate and geometry distortion, such that the compression flexibility and performance can be greatly improved. Experimental results show the interest of the proposed multi-scale method for lossy compression of geometry.

Social difficulties are the hallmark features of Autism Spectrum Disorder (ASD) and can lead to atypical visual attention towards stimuli. Eye movements encode rich information about attention and psychological factors of an individual, which could help to characterize the traits of ASD. Learning atypical eye movements of the individuals with ASD towards various stimuli is important and has many application scenarios. However, due to the lack of open datasets, research in this sense is still limited. In this work, we present an open dataset of eye movements of children with Autism Spectrum Disorder. It consists of 300 natural scene images and the corresponding eye movement data collected from 14 children with ASD and 14 healthy controls. In particular, fixation maps and scanpaths are available in the dataset. Based on this dataset, researchers could analyze the visual traits of children with ASD and design specialized visual attention models to promote research in related fields, as well as design specialized models to identify the individuals with ASD. The dataset can be accessed in http://doi.org/10.5281/zenodo.2647418

International audience; Different orientations of rays emitted by a 3D scene are captured in Light field (LF) imaging. In this, spatial along with angular information of the scene is captured. LF offers higher dimensional representation which needs advanced processing such as post-capture refocusing, depth sensing, compression, and reconstruction. These processing methods often induce artifacts as the quality is degraded. So, it is important to understand the mechanism of human visual quality perception to find the optimal performance that maximizes the quality of experience (QoE) of LF. The measurement of this data degradation is possible with the availability of a suitable dataset. In this paper, we present a comprehensive overview and discussion of publicly available LF datasets with emphasis on dataset with subject score. Furthermore, we discuss the current trends and existing challenges in LF dataset creation for QoE.

Data size is the bottleneck for developing deep saliency models, because collecting eye-movement data is very time-consuming and expensive. Most of current studies on human attention and saliency modeling have used high-quality stereotype stimuli. In real world, however, captured images undergo various types of transformations. Can we use these transformations to augment existing saliency datasets? Here, we first create a novel saliency dataset including fixations of 10 observers over 1900 images degraded by 19 types of transformations. Second, by analyzing eye movements, we find that observers look at different locations over transformed versus original images. Third, we utilize the new data over transformed images, called data augmentation transformation (DAT), to train deep saliency models. We find that label-preserving DATs with negligible impact on human gaze boost saliency prediction, whereas some other DATs that severely impact human gaze degrade the performance. These label-preserving valid augmentation transformations provide a solution to enlarge existing saliency datasets. Finally, we introduce a novel saliency model based on generative adversarial networks (dubbed GazeGAN). A modified U-Net is utilized as the generator of the GazeGAN, which combines classic "skip connection" with a novel "center-surround connection" (CSC) module. Our proposed CSC module mitigates trivial artifacts while emphasizing semantic salient regions, and increases model nonlinearity, thus demonstrating better robustness against transformations. Extensive experiments and comparisons indicate that GazeGAN achieves state-of-the-art performance over multiple datasets. We also provide a comprehensive comparison of 22 saliency models on various transformed scenes, which contributes a new robustness benchmark to saliency community. Our code and dataset are available at.

International audience; In this paper a method is proposed for estimating the orientation of industrial parts. Classical 2d images of the part are used to train a deep neural network and infer the part pose by computing its quaternion representation. Another innovative point of this work is in the use of synthetic data which are generated on the fly during the network training from a textured cad model placed in a virtual scene. This way overcomes the difficulties to obtain pose ground truth images from real images. At the same time, using the cad model with several lighting conditions and material reflectances allows to anticipate challenging industrial situations. As a first step of the method, the part is separated from the background using a semantic segmentation network. Then a depth image of the part is produced employing an encoder-decoder network with skip connections. Finally, the input image is associated with its depth map to estimate the part orientation with a fully connected network using a geodesic loss function. This method can estimate the part pose from a rendered image with a 5° accuracy while inferring from real images gives visually convincing results suitable for any pose refinement processes.

View synthesis is the most important technique in multi-view and free-viewpoint videos. The whole view synthesis includes the acquisition and processing of texture and depth images, and the virtual view rendering stage. Existing quality metrics for view synthesis have limited ability for the whole synthesis process for the following reasons. First, they are dedicated to a single stage of view synthesis, overlooking the commonality of all the possible distortions introduced in the whole process. Moreover, they only extract low-level features for quality assessment, ignoring the perceptual degradation caused by the mid-level contours that are destructed by heavy distortions in texture/depth images and the imperfect view rendering, which represent the spatial distribution/connection of adjacent contour pixels. Inspired by the above facts, this paper presents a quality metric for view synthesis using both Low-level and Mid-level Structural representation (LMS), aiming to accurately evaluate the distortions in the whole view synthesis process. Specifically, the scale space is first constructed to mimic the hierarchical property of the human visual system. Then, the statistics of gradient orientation is integrated with the statistics of gradient intensity for the low-level structural representation, which is motivated by the importance of the orientation selectivity mechanism to visual perception. Further, the mid-level structure is represented using bag of words for contour description based on the sparse coding of the primary visual cortex. Then the distances of both the low-level and mid-level features between the synthesized and reference images are calculated. Finally, two distances are integrated to generate the whole quality score. Extensive experiments on two public view synthesis databases demonstrate the superiority of the proposed method to the state-of-the-arts in evaluating the quality of the whole view synthesis.

In image/video systems, contrast adjustment which manages to enhance visual quality is nowadays an important research topic. Yet very limited struggles have been made to the exploration of visual quality assessment for contrast adjustment. To tackle the issue, this paper proposes a novel reduced-reference (RR) quality metric with the integration of bottom-up and top-down strategies. The former one stems from the recently revealed free energy principle that tells that the human visual system seeks to comprehend an input image via uncertainty removal, while the latter one is toward using the symmetric Kullback–Leibler divergence to compare the histogram of the contrast-altered image with that of the pristine image. The bottom-up and top-down strategies are lastly incorporated to derive the RR contrast-altered image quality measure. A comparison using numerous existing IQA models is carried out on five contrast related databases/subsets in CID2013, CCID2014, CSIQ, TID2008, and TID2013, and experimental results validate the superiority of the proposed technique.

International audience; Virtual reality (VR) headsets offer a large and immersive workspace for displaying visualizations with stereoscopic vision, as compared to traditional environments with monitors or printouts. The controllers for these devices further allow direct three-dimensional interaction with the virtual environment. In this paper, we make use of these advantages to implement a novel multiple and coordinated view (MCV) system in the form of a vertical stack, showing tilted layers of geospatial data. In a formal study based on a use-case from urbanism that requires cross-referencing four layers of geospatial urban data, we compared it against more conventional systems similarly implemented in VR: a simpler grid of layers, and one map that allows for switching between layers. Performance and oculometric analyses showed a slight advantage of the two spatial-multiplexing methods (the grid or the stack) over the temporal multiplexing in blitting. Subgrouping the participants based on their preferences, characteristics, and behavior allowed a more nuanced analysis, allowing us to establish links between e.g., saccadic information, experience with video games, and preferred system. In conclusion, we found that none of the three systems are optimal and a choice of different MCV systems should be provided in order to optimally engage users.

The appropriate characterization of the test material, used for subjective evaluation tests and for benchmarking image and video processing algorithms and quality metrics, can be crucial in order to perform comparative studies that provide useful insights. This paper focuses on the characterisation of 360-degree images. We discuss why it is important to take into account the geometry of the signal and the interactive nature of 360-degree content navigation, for a perceptual characterization of these signals. Particularly, we show that the computation of classical indicators of spatial complexity, commonly used for 2D images, might lead to different conclusions depending on the geometrical domain used to represent the 360-degree signal. Finally, new complexity measures based on the analysis of visual attention and content exploration patterns are proposed.

International audience; Recent studies brought to light that the semantic labels (e.g. Excellent, Good, Fair, Poor, and Bad) commonly associated with discrete scale ITU subjective quality evaluation induce a bias in MOS computation and that such a bias can be quantified by some reference coefficients which are independent with respect to the observers panel. The present paper reconsiders these results from a standard upgrading perspective. First, it theoretically investigates the way in which results obtained on semantically labeled scales can be “cleaned” from such an influence and derives the underlying computation formula for the mean opinion score. Secondly, it suggests a unitary evaluation procedure featuring both semantic free MOS computation and backward compatibility with respect to state-of-the-art solutions. The theoretical and methodological results are supported by subjective experiments corresponding to a total of 440 human observers, alternatively scoring 2D and stereoscopic video content. For each type of content, both high and low quality excerpts are alternatively considered. For each type of content and for each type of quality a 5 level (Excellent, Good, Fair, Poor, and Bad) grading scales is considered.

International audience; Eye tracking constitutes a valuable tool for the examination of human visual behavior since it provides objective measurements related to the performed visual strategies during the observation of any type of visual stimuli. Over the last decade, eye movement analysis contributed substantially to the better understanding of how visual attention processes work in different types of maps. Considering the clear need for the examination of map user reaction during the observation of realistic cartographic products (i.e. static maps, animated maps, interactive and multimedia maps), a critical amount of experimental studies were performed in order to study different aspects related to map reading process by the cartographic community. The foundation of these studies is based on theories and models that have been developed in similar research domains (i.e. psychology, neuroscience etc.), while the research outcomes that produced over these years can be used directly for the design of more effective and efficient maps. The aim of the present article is to summarize and present the current panorama of the existing eye tracking studies in cartographic research appeared over the last decade. Additionally, methodological contributions (including analysis tools) of cartographic society in the field of eye movement analysis are reported, while existing challenges and future perspectives are also discussed.

International audience; With the ongoing advances in Light Field(LF) technology, research in LF acquisition, compression, processing has gained momentum. This increased the need for objective quality evaluation of LF content. Many processing algorithms are still optimized against peak signal to noise ratio(PSNR). Lately, several attempts have been made to improve objective quality evaluation such as extending 2D metrics to 4D LF domain. However, there is still a great room for improvement. In this paper, we experiment with existing 2D image quality metrics on the Epipolar Plane Image representations of LF content to reveal characteristics of LF related distortions. We discuss the challenges and suggest possible directions towards a LF image quality evaluation on EPI representations.

This paper proposes to blindly evaluate the quality of images synthesized via a depth image-based rendering (DIBR) procedure. As a significant branch of virtual reality (VR), superior DIBR techniques provide free viewpoints in many real applications, including remote surveillance and education; however, limited efforts have been made to measure the performance of DIBR techniques, or equivalently the quality of DIBR-synthesized views, especially in the condition when references are unavailable. To achieve this aim, we develop a novel blind image quality assessment (IQA) method via multiscale natural scene statistical analysis (MNSS). The design principle of our proposed MNSS metric is based on two new natural scene statistics (NSS) models specific to the DBIR-synthesized IQA. First, the DIBR-introduced geometric distortions damage the local self-similarity characteristic of natural images, and the damage degrees of self-similarity present particular variations at different scales. Systematically combining the measurements of the variations mentioned above can gauge the naturalness of the input image and thus indirectly reflect the quality changes of images generated using different DIBR methods. Second, it was found that the degradations in main structures of natural images at different scales remain almost the same, whereas the statistical regularity is destroyed in the DIBR-synthesized views. Estimating the deviation of degradations in main structures at different scales between one DIBR-synthesized image and the statistical model, which is constructed based on a large number of natural images, can quantify how a DIBR method damages the main structures and thus infer the image quality. Via trials, the two NSS-based features extracted above can well predict the quality of DIBR-synthesized images. Further, the two features come from distinct points of view, and we hence integrate them via a straightforward multiplication to derive the proposed blind MNSS metric, which achieves better performance than each component and state-of-the-art quality methods.

Quality of Experience of multimedia content is investigated to understand the overall acceptability of the content as perceived by the end user. Recent perspectives stress the need to focus not only on perceptual aspects of the content but also on the interaction between perception and cognitive–emotional aspects. Our contribution lies on this direction by presenting a contactless approach to estimate heart rate variability by evaluating face skin color variations in a video sequence. The proposed framework allows to analyze long video sequences depicting a subject and it is robust with respect to small movements and illumination changes. The system is validated on a dataset of 64 sequences where users observe 3D movies selected to induce heart rate variations.

Tone-mapping is a crucial step in the task towards displaying high dynamic range (HDR) images on standard displays. Given the number of possible ways to tone-map such images, development of an objective quality criterion, enabling selection of the most suitable tone-mapping operator (TMO) and setting its parameters in order to maximize the quality of the reproduction, is of high interest. In this paper, a new objective metric for natural tone-mapped images is proposed. It is based on a fusion of several perceptually relevant features that have been carefully selected using an appropriate feature selection procedure. The outcome of the selection also provides a valuable insight into the importance of particular perceptual aspects when judging the quality of tone-mapped HDR content. The performance of the resulting combination of features is thoroughly evaluated with respect to three publicly available databases and compared to several relevant state-of-the-art criteria. The proposed approach is shown to significantly outperform the tested metrics and can, therefore, be considered a competitive alternative for tone-mapped images evaluation.

International audience; Visual field defects are a world-wide concern, and the proportion of the population experiencing vision loss is ever increasing.Macular degeneration and glaucoma are among the four leading causes of permanent vision loss.Identifying and characterising visual field losses from gaze alone could prove crucial in the future for screening tests, rehabilitation therapies and monitoring.In this experiment, 54 participants took part in a free-viewing task of visual scenes while experiencing artificial scotomas (central and peripheral) of varying radii in a gaze-contingent paradigm.We studied the importance of a set of gaze features as predictors to best differentiate between artificial scotoma conditions.Linear mixed models were utilised to measure differences between scotoma conditions.Correlation and factorial analyses revealed redundancies in our data.Finally, hidden Markov models and recurrent neural networks were implemented as classifiers in order to measure the predictive usefulness of gaze features.The results show separate saccade direction biases depending on scotoma type.We demonstrate that the saccade relative angle, amplitude and peak velocity of saccades are the best features on the basis of which to distinguish between artificial scotomas in a free-viewing task.Finally, we discuss the usefulness of our protocol and analyses as a gaze-feature identifier tool that discriminates between artificial scotomas of different types and sizes.

International audience; Background: Individuals with schizophrenia have difficulty in extracting salient information from faces. Eye-tracking studies have reported that these individuals demonstrate reduced exploratory viewing behaviour (i.e.reduced number offixations and shorter scan paths) compared to healthy controls. Oxytocin has previouslybeen demonstrated to exert pro-social effects and modulate eye gaze during face exploration. In this study, wetested whether oxytocin has an effect on visual attention in patients with schizophrenia.Methods: Nineteen male participants with schizophrenia received intranasal oxytocin 40UI or placebo in adouble-blind, placebo-controlled, crossover fashion during two visits separated by seven days. They engagedin a free-viewing eye-tracking task, exploring images of Caucasian men displaying angry, happy, and neutralemotional expressions; and control images of animate and inanimate stimuli. Eye-tracking parameters included:total number offixations, mean duration offixations, dispersion, and saccade amplitudes.Results: We found a main effect of treatment, whereby oxytocin increased the total number offixations, dispersion, and saccade amplitudes, while decreasing the duration offixations compared to placebo. This effect, how-ever, was non-specific to facial stimuli. When restricting the analysis to facial images only, we found the sameeffect. In addition, oxytocin modulatedfixation rates in the eye and nasion regions.Discussion: This is thefirst study to explore the effects of oxytocin on eye gaze in schizophrenia. Oxytocin had enhanced exploratory viewing behaviour in response to both facial and inanimate control stimuli. We suggest thatthe acute administration of intranasal oxytocin may have the potential to enhance visual attention inschizophrenia.

Machine learning (ML) is an essential part of recent advances in computer science. To fully exploit its potential, ML-based algorithms require a considerable amount of annotated data to be used for training. This represents a severe limitation in the field of image and video quality assessment since obtaining large-scale annotated databases is time-consuming and expensive. Moreover, the resulting quality estimators are mainly restricted only to the usecases included in the dataset used for their training. This paper proposes a strategy allowing for combination of multiple databases for training of objective image and video quality assessment algorithms. Using this strategy, the algorithms can be trained using all of the existing relevant databases together which allows to increase the amount of data-points and usecases in orders of magnitude. The potential of the proposed method is demonstrated by re-training the combination of features from Video Multimethod Assessment Fusion (VMAF) algorithm resulting in the significant improvement of its performance with respect to 20 video databases.

Exploring unexploited but newly digitized resources to find relevant information is a complicated task due to the amount of available resources. Thanks to the ANR project CIRESFI, the most important resource for the Italian Comedy of the 18th century, is a set of accounting registers consisting of 28,000 pages. Information retrieval is a long and complex process that requires expertise at every step: detection and segmentation in paragraphs, lines or words, features extraction, handwriting recognition. Systems based on deep neural networks dominate these approaches. The major issue is the need of a large amount of data to achieve their learning. However, the registers of the Italian Comedy have no ground truth. To overcome this lack of data, we explore approaches that involving transfer learning. That means using heterogeneous labeled and available data, with at least one common feature with our data to drive the systems, and then applying them to our data. All of our experiments have shown us the difficulty of carrying out this task, each choice at each stage having a strong impact on the rest of the system. We converge on a solution separating the optical model from the language model in order to achieve independent learning with different available resources and joining together thanks to a projection of the information into a non-latent common space.; La tâche d'exploration dans des ressources inexploitées mais nouvellement numérisées, afin d'y trouver des informations pertinentes, est complexifiée par la quantité de ressources disponibles. Grâce au projet ANR CIRESFI, la ressource la plus importante, pour la Comédie-Italienne du XVIIIe siècle, est un ensemble de registres comptables constituée de 28 000 pages. L'extraction d'informations est un processus long et complexe qui demande une expertise à chaque étape : détection et segmentation, extraction de caractéristiques, reconnaissance d’écriture manuscrite. Les systèmes à base de réseaux de neurones profonds dominent dans l'ensemble ces approches. Le problème majeur est qu'ils nécessitent d'avoir une grande quantité de données pour réaliser leur apprentissage. Cependant, les registres de la Comédie-Italienne ne possèdent pas de vérité terrain. Pour palier ce manque de données, nous explorons des approches pouvant opérer un apprentissage par transfert de connaissance. Cela signifie utiliser un ensemble de données déjà étiquetées et disponibles, possédant un minimum de points communs avec nos données pour entraîner les systèmes, pour ensuite les appliquer sur nos données. L'ensemble de nos expérimentations nous ont montré la difficulté de réaliser cette tâche, chaque choix à chaque étape ayant un impact fort sur la suite du système. Nous convergeons vers une solution séparant le modèle optique du modèle de langage afin de réaliser un apprentissage indépendant avec différents types de ressources disponibles et se rejoignant grâce à une projection de l'ensemble des informations dans un espace commun non-latent.

Exploring unexploited but newly digitized resources to find relevant information is a complicated task due to the amount of available resources. Thanks to the ANR project CIRESFI, the most important resource for the Italian Comedy of the 18th century, is a set of accounting registers consisting of 28,000 pages. Information retrieval is a long and complex process that requires expertise at every step: detection and segmentation in paragraphs, lines or words, features extraction, handwriting recognition. Systems based on deep neural networks dominate these approaches. The major issue is the need of a large amount of data to achieve their learning. However, the registers of the Italian Comedy have no ground truth. To overcome this lack of data, we explore approaches that involving transfer learning. That means using heterogeneous labeled and available data, with at least one common feature with our data to drive the systems, and then applying them to our data. All of our experiments have shown us the difficulty of carrying out this task, each choice at each stage having a strong impact on the rest of the system. We converge on a solution separating the optical model from the language model in order to achieve independent learning with different available resources and joining together thanks to a projection of the information into a non-latent common space.; La tâche d'exploration dans des ressources inexploitées mais nouvellement numérisées, afin d'y trouver des informations pertinentes, est complexifiée par la quantité de ressources disponibles. Grâce au projet ANR CIRESFI, la ressource la plus importante, pour la Comédie-Italienne du XVIIIe siècle, est un ensemble de registres comptables constituée de 28 000 pages. L'extraction d'informations est un processus long et complexe qui demande une expertise à chaque étape : détection et segmentation, extraction de caractéristiques, reconnaissance d’écriture manuscrite. Les systèmes à base de réseaux de neurones profonds dominent dans l'ensemble ces approches. Le problème majeur est qu'ils nécessitent d'avoir une grande quantité de données pour réaliser leur apprentissage. Cependant, les registres de la Comédie-Italienne ne possèdent pas de vérité terrain. Pour palier ce manque de données, nous explorons des approches pouvant opérer un apprentissage par transfert de connaissance. Cela signifie utiliser un ensemble de données déjà étiquetées et disponibles, possédant un minimum de points communs avec nos données pour entraîner les systèmes, pour ensuite les appliquer sur nos données. L'ensemble de nos expérimentations nous ont montré la difficulté de réaliser cette tâche, chaque choix à chaque étape ayant un impact fort sur la suite du système. Nous convergeons vers une solution séparant le modèle optique du modèle de langage afin de réaliser un apprentissage indépendant avec différents types de ressources disponibles et se rejoignant grâce à une projection de l'ensemble des informations dans un espace commun non-latent.

Proprietor: Université de Nantes et al; The present invention relates generally to the field of video compression and more specifically to a method for encoding video frames based on a local texture synthesis

The training and performance analysis of objective video quality assessment algorithms is complex due to the huge variety of possible content classes and transmission distortions. Several secondary issues such as free parameters in machine learning algorithms and alignment of subjective datasets put an additional burden on the developer. In this paper, three subsequent steps are presented to address such issues. First, the content and coding parameter space of a large-scale database is used to select dedicated subsets for training objective algorithms. This aims at providing a method for selecting the most significant contents and coding parameters from all imaginable combinations. In the practical case where only a limited set is available, it also helps us to avoid redundancy in the training subset selection. The second step is a discussion on performance measures for algorithms that employ machine-learning methods. The particularity of the performance measures is that the quality of the training and verification datasets is taken into consideration. Common issues that often use existing measures are presented, and improved or complementary methods are proposed. The measures are applied to two examples of no-reference objective assessment algorithms using the aforementioned subsets of the large-scale database. While limited in terms of practical applications, this sandbox approach of objectively predicting an objectively evaluated video sequences allows for eliminating additional influence factors from subjective studies. In the third step, the proposed performance measures are applied to the practical case of training and analyzing assessment algorithms on readily available subjectively annotated image datasets. The presentation method in this part of the paper can also be used as an exemplified recommendation for reporting in-depth information on the performance. Using this presentation method, future publications presenting newly developed quality assessment algorithms may be significantly improved.

International audience; This paper deals with the current trends of technologies which aim at representing efficiently 3D point cloud, they have become a popular challenge for processing, storing and conveying the data independently of how it was captured. Compressing attributes of 3D point clouds such as geometry, colours or normal directions remain challenging problem, since these signals are unstructured. We propose an adaptive Tree-Structured Point-Lattice Vector Quantization (TSPLVQ), resulting in hierarchically structured 3D content, to improve compression performance for static point cloud.The novelty of the proposed approach lies in adaptively selecting the optimal quantization scheme for the point cloud geometric data, such that its intrinsic correlations can be better exploited.Two quantization modes are dedicated to project recursively the 3D point clouds into a series of embedded truncated cubic lattices. At each step of the process, the optimal quantization mode is chosen according to a rate-distortion criterion in order to achieve the best trade-off between coding rate and geometry distortion, such that the compression flexibility and performance can be greatly improved. To achieve rate-distortion optimization, the core of the proposed method relies on an accurate estimation of the distortion inside each cubic voronoï, between fitting plane and cloud points, taking into account the visual rendering of the 3D object. We experimentally evaluated the interest of the proposed method for geometry compression and visualization tasks.

Research on understanding how people watch/explore Virtual Reality (VR) and 360° content is crucial to develop appropriate technologies for processing, encoding, delivering and rendering media content in order to provide high-quality immersive experiences to users. In this sense, the work described in this paper aims at supporting the research on visual attention modeling for omnidirectional content, by presenting a dataset and toolbox that were created for the Grand Challenge “Salient360!” organized at ICME’17. In particular, the dataset includes 360° images and their corresponding saliency maps and scanpaths, obtained from a subjective experiment in which eye gazes and head movements of observers were recorded during a free-viewing task. In addition, guidelines to obtain saliency maps and scanpaths from eye and head tracking data for omnidirectional images are provided. Finally, together with the dataset, a toolbox has been made publicly available, containing scripts for comparing saliency maps and scanpaths to benchmark developed models.

International audience; Long short-term memory networks (LSTM) achieves great success in temporal dependency modelling for chain-structured data, such as texts and speeches. An extension towards more complex data structures as encountered in 2D graphic languages is proposed in this work. Specifically, we address the problem of handwritten mathematical expression recognition, using a Tree-based BLSTM architecture allowing the direct labelling of nodes (symbol) and edges (relationship) from a graph modelling the input strokes. One major difference with the traditional approaches is that there is no explicit segmentation, recognition and layout extraction steps but a unique trainable system that produces directly a Stroke Label Graph describing a mathematical expression. The proposed system, considering no grammar, achieves competitive results in online math expression recognition domain.

The present article describes an experimental study towards the examination of human visual behavior during the observation of unmanned aerial vehicles (UAVs) videos. Experimental performance is based on the collection and the quantitative &amp, qualitative analysis of eye tracking data. The results highlight that UAV flight altitude serves as a dominant specification that affects the visual attention process, while the presence of sky in the video background seems to be the less affecting factor in this procedure. Additionally, the main surrounding environment, the main size of the observed object as well as the main perceived angle between UAV’s flight plain and ground appear to have an equivalent influence in observers’ visual reaction during the exploration of such stimuli. Moreover, the provided heatmap visualizations indicate the most salient locations in the used UAVs videos. All produced data (raw gaze data, fixation and saccade events, and heatmap visualizations) are freely distributed to the scientific community as a new dataset (EyeTrackUAV) that can be served as an objective ground truth in future studies.

Light Field imaging is a promising technology that allows to capture the whole set of light rays in a scene thus enabling the generation of perspective views from any position. This possibility can be exploited in several application scenarios, such as virtual and augmented reality or depth estimation. In this framework many issues arise due different aspects such as the large amount of generated data or to the need of dedicated and expensive hardware for Light Field capturing. Moreover, the Light Field carries information about the entire scene and the data that is delivered to the users largely differs from the traditional 2D and 3D media in terms of content and way of fruition. Dedicated rendering technology and devices for the Light Field are nowadays still not mature or quite expensive and the best option is to render the Light Field data on a conventional 2D screen. Consequently, there is the need for finding the best visualization technique that allows to exploit the information in the Light Field while being accepted by the viewers. In this paper we address this issue by considering six visualization options and by running experimental tests to study which is the technique preferred by the users.

International audience; Lack of data can be an issue when beginning a new study on historical handwritten documents. In order to deal with this, we present the character-based decoder part of a multilingual approach based on transductive transfer learning for a historical handwriting recognition task on Italian Comedy Registers. The decoder must build a sequence of characters that corresponds to a word from a vector of letter-ngrams. As learning data, we created a new dataset from untapped resources that covers the same domain and period of our Italian Comedy data, as well as resources from common domains, periods, or languages. We obtain a 97.42% Character Recognition Rate and a 86.57% Word Recognition Rate on our Italian Comedy data, despite a lexical coverage of 67% between the Italian Comedy data and the training data. These results show that an efficient system can be obtained by a carefully selecting the datasets used for the transfer learning.

International audience; Lack of data can be an issue when beginning a new study on historical handwritten documents. To deal with this, we propose a deep-learning based recognizer which separates the optical and the language models in order to train them separately using different resources. In this work, we present the optical encoder part of a multilingual transductive transfer learning applied to historical handwriting recognition. The optical encoder transforms the input word image into a non-latent space that depends only on the letter-n-grams: it enables it to be independent of the language. This transformation avoids embedding a language model and operating the transfer learning across languages using the same alphabet. The language decoder creates from a vector of letter-n-grams a word as a sequence of characters. Experiments show that separating optical and language model can be a solution for multilingual transfer learning.

Assessment of multimedia quality relies heavily on subjective assessment, and is typically done by human subjects in the form of preferences or continuous ratings. Such data are crucial for analysis of different multimedia-processing algorithms as well as validation of objective (computational) methods for the said purpose. To that end, statistical testing provides a theoretical framework toward drawing meaningful inferences, and making well-grounded conclusions and recommendations. While parametric tests (such as $t$ test, ANOVA, and error estimates like confidence intervals) are popular and widely used in the community, there appears to be a certain degree of confusion in the application of such tests. Specifically, the assumptions of normality and homogeneity of variance are often not well understood, leading to incorrect application and/or interpretation of the statistical test results. Therefore, the main goal of this paper is to present new guidelines toward proper use of statistical tests and, hence, fix some of the issues in multimedia quality assessment. The said guidelines are derived based on theoretical analysis of sampling distribution of test statistics, and consider practical aspects of data analysis in the said domain. Experimental results on both simulated and real data are presented to support the arguments made. Software that implements the said recommendations is also made publicly available, in order to help researchers and practitioners perform correct statistical comparison of models.

Image stitching-composition of different viewpoint images to form a 360-degree panoramic image-is an essential component towards immersive applications like VR and AR. A no-reference (NR) quality metric specifically designed to evaluate stitched panoramic images is highly desirable when ground-truth reference images are not available. In this paper, we use Convolutional Sparse Coding (CSC) with a set of convolutional filters to locate stitching-specific distortions in a target image, and design trained kernels to quantify the compound effects of multiple distortion types in a local region. Specifically, our contributions are: i) a training database labeled with location information of the distortion regions is released; ii) a NR metric is proposed to accurately assess stitching-specific artifacts like ghosting using convolutional sparse coding; and iii) a novel sequential feature selection algorithm is proposed to quantify the aforementioned compound distortion effects. In extensive experiments, we show that the performance of our proposed NR metric is comparable to the state-of-the-art full-reference metrics designed for stitched images.

International audience

International audience; Saliency models aim to predict where we look at within a visual scene. They are based on low-level visual features including color, intensity, and orientation. They process these visual features to generate a 2D saliency map, indicating the most salient part of the image. In contrast with saliency models, saccadic models intend to predict the sequence of eye fixations, i.e. the way observers deploy their gaze while viewing a stimulus on screen. Rather than computing a unique saliency map, saccadic models compute plausible visual scanpaths, i.e. the actual sequence of fixations and saccades unfolding across time. By the term plausible, we mean that the predicted scanpaths should be as similar as possible to human scanpaths. They should exhibit similar characteristics, such as the same distributions of saccade amplitudes and saccade orientations. Saccadic models offer many advantages over saliency models. The most important one is the ability to tailor the saccadic model to a particular context, such as a particular type of scene, a particular population or to a particular task at hand. This modelling framework not only relies on the image salience (i.e. properties of the stimulus) but most importantly on our ability to model viewing biases and systematic gaze patterns of observers. In this presentation, we introduce an age-dependent saccadic model able to reproduce the gaze behavior of a certain target age group.

National audience; L'absence de données annotées peut être une difficulté majeure lorsque l'on s'intéresse à l'analyse de documents manuscrits anciens. Pour contourner cette difficulté, nous proposons de diviser le problème en deux, afin de pouvoir s'appuyer sur des données plus facilement accessibles. Dans cet article nous présentons la partie décodeur d'un encodeur-décodeur multimodal utilisant l'apprentissage par transfert de connaissances pour la transcription des titres de pièces de la Comédie Italienne. Le décodeur transforme un vecteur de n-grammes au niveau caractères en une séquence de caractères correspondant à un mot. L'apprentissage par transfert de connaissances est réalisé principalement à partir d'une nouvelle ressource inexploitée contemporaine à la Comédie-Italienne et thématiquement proche ; ainsi que d'autres ressources couvrant d'autres domaines, des langages différents et même des périodes différentes. Nous obtenons 97,27% de caractères bien reconnus sur les données de la Comédie-Italienne, ainsi que 86,57% de mots correctement générés malgré une couverture de 67,58% uniquement entre la Comédie-Italienne et l'ensemble d'apprentissage. Les expériences montrent qu'un tel système peut être une approche efficace dans le cadre d'apprentissage par transfert. ABSTRACT Neural decoder for the transcription of historical handwritten documents. The lack of data can be an issue at the beginning of a study on new historical handwritten documents. To solve this issue, we present the decoder part of a multimodal approach based on transductive transfer learning for transcripting play titles of the Italian Comedy. MOTS-CLÉS : modèle neuronal, apprentissage par transfert, transcription, Comédie Italienne.

Virtual Reality (VR) provides the users with new immersive media experiences, offering the possibility to freely explore 360° content. Understanding these new exploration behaviors is crucial for the development of efficient techniques for processing, coding, delivering and rendering omnidirectional content to offer the highest possible Quality of Experience (QoE). Progress has already been made on visual attention (VA) modeling for 360° content. In this paper we briefly review the current status of research on this topic that led us to propose a benchmarking platform for evaluating and comparing the performance of models for saliency and scanpath prediction for 360° content. This paper introduces the ‘UN Salient360! benchmark” platform featuring a dataset, a toolbox and a framework for evaluation of different class of models. This online platform can be found in httns://salient360.ls2n.fr/.

International audience; Background: Deficits in social cognition often develop during the prod- romal stages of psychosis, remain stable over the course of the illness, and have a dramatic impact on daily functioning (Fett et al., 2011). Social cue processing, particularly face perception, plays a critical role in social cognitive functioning. Patients with schizophrenia struggle to extract information from faces and interpret facial expressions (Kohler et al., 2010). These deficits may be explained by restricted visual atten- tion. Indeed, eye-tracking studies have demonstrated that people with schizophrenia show reduced exploratory behaviour (i.e. reduced number of fixations and longer fixation durations) in response to facial stimuli compared to healthy controls (e.g. Manor et al., 1999). Oxytocin has been demonstrated to exert pro-social effects on behaviour and modu- late eye gaze during perception of faces. In the present study, we tested whether the neuropeptide, oxytocin, has a compensatory effect on visual processing of human faces.Methods: Twenty right-handed male subjects with schizophrenia (n = 16) or schizoaffective disorder (n = 4) were administered intranasal oxytocin 40UI or placebo in a double-blind, placebo-controlled, cross- over fashion during two visits separated by 7 days. Participants engaged in a free-viewing eye-tracking task, during which they were looking at 6 facial images of two Caucasian men displaying angry, happy, and neutral facial expressions, and 6 control images in a random order. Eye- tracking measures including 1) total number of fixations, 2) dispersion, 3) saccade amplitude, and 4) mean duration of fixations were captured using the EyeLink 1000 system (SR Research Ltd, Ottawa, Ontario, Canada). Four separate 2 x 4 repeated-measures analysis of variance (ANOVA) were carried out to evaluate the within-subject effects of treatment, stimuli, and the interactions between stimuli and treatment (p < .05, two-tailed).Results: We found a main effect of treatment (F1,17 = 16.139, p = .001), but not a main effect of stimuli (F3,51 = 1.479, p > .231) on total number of fixations. There was a main effect of treatment on duration of fixation, (F1,13 = 5.455, p = .036) but not a main effect of stimuli (F3,39 = 1.267, p = .299). For dispersion, there was a significant main effect of stimuli (F3,51 = 3.424, p = .024) but no main effect of treatment (F1,17 = 3.170, p = .093). Analysis of saccade amplitudes revealed no main effect of treat- ment (F1,17 = 2.666, p = .121) or stimuli (F3,51 = 0.289, p = .833). None of the interactions reached significance.Discussion: To our knowledge, this is the first study to explore the effects of oxytocin on eye movements in individuals with schizophre- nia. We found that oxytocin increased exploratory viewing behaviour in response to affective facial stimuli by significantly increasing the total number and duration of fixations compared to placebo. While previous findings regarding oxytocin have been inconsistent, our findings are in line with research showing that the intranasal administration of 40UI oxytocin may improve social cognitive deficits in schizophrenia (e.g. Davis et al., 2013). Future experiments may wish to explore the corre- lation between eye movement changes induced by oxytocin and facial affect recognition in larger samples.

International audience

International audience

International audience; By bridging uncompressed-domain saliency detection and MPEG-4 AVC compression principles, the present paper advances a methodological framework for extracting the saliency maps directly from the stream syntax elements. In this respect, inside each GOP, the intensity, color, orientation and motion elementary saliency maps are related to the energy of the luma coefficients, to the energy of chroma coefficients, to the gradient of the prediction modes and to the amplitude of the motion vectors, respectively. The three spatial saliency maps are pooled according to an average formula, while the static-temporal fusion is achieved by six different formulas. The experiments consider both ground-truth and applicative evaluations. The ground-truth benchmarking investigates the relation between the predicted MPEG-4 AVC saliency map and the actual human saliency, captured by eye-tracking devices. It is based on two corpora (representing density fixation maps and saccade locations), two objective criteria (related to the closeness between the predicted and the real saliency maps and to the difference between the behavior of the predicted saliency map in fixation and random locations), two objective measures (KLD – the Kullback Leibler Divergence and AUC – the Area Under the ROC Curve) and 5 state-of-the-art saliency models (3 acting in spatial domain and 2 acting in compressed domain). The applicative validation is carried out by integrating the MPEG-4 AVC saliency map into a robust watermarking application. As an overall conclusion, the paper demonstrates that although the MPEG-4 AVC standard does not explicitly relies on any visual saliency principle, its stream syntax elements preserve this property. Four main benefits for the MPEG-4 AVC based saliency extraction are thus brought to light: (1) it outperforms (or, at least, is as good as) state-of-the-art uncompressed domain methods, (2) it allows significant gains to be obtained in watermarking transparency (for prescribed data payload and robustness), (3) it is less sensitive to the randomness in the processed visual content, and (4) it has a linear computational complexity. For instance, the ground truth results exhibit absolute relative gains between 60% and 164% in KLD, between 17% and 21% in AUC, and relative gains in KLD sensitivity between 1.18 and 6.12 and in AUC sensitivity between 1.06 and 33.7; the applicative validation brings to light transparency gains up to 10 dB in PSNR.

International audience

International audience

Purpose Abnormal torsion could be associated with cyclovertical strabismus, but torsion measurements are not reliable in children. To assess an objective fundus torsion evaluation in a paediatric population, we used Non-Mydriatic Fundus photography (NMFP) in healthy and cyclovertical strabismus patients to evaluate the disc-foveal angle over time and observers. Methods We used a retrospective set of NMFP including 24 A or V-pattern strabismus and 27 age-matched normal children (mean age 6.4 and 6.7 years respectively), taken during 2 distinct follow-up consultations (separated by 251 and 479 days respectively). Each disc-foveal angle measurement (from which the ocular torsion can be assessed) was performed by 5 different observers, using graphical software and based on reproducible fundus anatomical marks. Statistical analysis was performed with a multivariate ANOVA using group, time and observers as factors, in addition to intraclass coefficient correlation (ICC) to assess measurement reproducibility. Results A significant difference of disc-foveal angle measures was observed between groups (p0,97). Conclusion Abnormal amount of objective torsion could be associated with alphabet-pattern strabismus. Disc-foveal angle evaluation by NMFP in a children population appears as a non-invasive, reliable and reproducible method.

Subjective assessment for image or video qualities is considered as the most reliable way to obtain the ground truth for the development of objective quality metrics, especially when leaded by Mean Opinion Score (MOS approaches). However, obtained MOS with standard protocols are noisy due to subject's personal characteristics, such as viewing experience, gender or profession, leading to uncertain ground truth driven by the number of panelists/subjects. The usual way to reduce uncertainty relies on raising this number. In this paper, we demonstrate how a recently introduced Maximum Likelihood Estimation (MLE) based quality recovery model can improve the discriminability of standard subjective quality assessment. Compared to straightforward MOS computation, we present a case study where one can save between 26% to 39% in terms of numbers of subjects at the same discriminability.

Machine learning has been attached greater attention in the field of quality assessment and related models have been developed by assuming that any region within images share the same quality score. However, this assumption may no longer stand when non-uniform distortions exist, e.g. geometric distortion within synthesized views in the case of free- view point video TV (FTV). In this paper, we explore to use Convolutional Sparse Coding (CSC), which computes a sparse representation for an entire image with the sum of a set of convolutions with dictionary filters instead of a linear combination of a set of dictionary atoms, to learn a visible codebook and propose a methodology to quantify the geometric distortion without using the reference. Experimental results show that the proposed no reference metric is the most visual friendly and reliable metric among the compared blind image metrics designed for synthesized images.

The assessment of Quality of Experience (QoE) for stereoscopic 3-D video is a challenging task, especially in 3-D video compression and transmission applications. The focus of this contribution is the development of a QoE assessment framework, in line with the latest standardization progress in the field of QoE assessment, for understanding the visual effect of asymmetric and symmetric encoding for immersive media. Asymmetric stereoscopic video coding exploits the binocular suppression of the human vision system by representing one of the two views with a lower quality. This processing, while of limited effects on image quality, may influence the overall QoE. Many studies show that the QoE of immersive media such as 3-DTV can be thought as the combination of the perceived visual quality, the perceived depth quality, the visual fatigue, and visual discomfort. In this paper, we aim at: 1) exploiting the concept of preference of experience and protocols recently standardized for characterizing QoE; 2) conducting a case study using these standardized protocols to investigate the factors involving visual discomfort in stereoscopic video sequences with a focus on binocular rivalry; and 3) presenting the results of subjective experiments performed, by using the perceptual quality and preference of experience assessment protocols, for evaluating the impact of symmetrical, asymmetrical, and alternate coding schemes.

This chapter is related to the concept of Quality of Experience (QoE) in multimedia applications. Throughout the chapter, QoE mostly has to be considered in such applied context, even if the term multimedia is not always mentioned. After reminding the concept of QoE, the chapter provides guidelines on: (1) how to design, conduct, and interpret QoE experiments with humans and (2) how to use the results obtained from such experiments for evaluating the performance of objective QoE estimators. The main factors influencing QoE are first summarized and the possibilities of involving particular aspects of QoE into the subjective tests are discussed. The most popular subjective procedures are introduced in detail, together with their advantages and weaknesses. Thorough instructions regarding processing the outcomes of the particular procedures are provided, as well as advanced methodologies for determining the significance of influence of the specific dimensions on the measured overall QoE. Finally, the possibilities of utilizing subjective data for evaluating the performance of objective QoE estimators are introduced in detail. Such methods are useful for comparison, training, and validation of the estimators.

International audience; In this work, we investigate handwriting recognition on new historical handwritten documents using transfer learning. Establishing a manual ground-truth of a new collection of handwritten documents is time consuming but needed to train and to test recognition systems. We want to implement a recognition system without performing this annotation step. Our research deals with transfer learning from heterogeneous datasets with a ground-truth and sharing common properties with a new dataset that has no ground-truth. The main difficulties of transfer learning lie in changes in the writing style, the vocabulary, and the named entities over centuries and datasets. In our experiment, we show how a CNN-BLSTM-CTC neural network behaves, for the task of transcribing handwritten titles of plays of the Italian Comedy, when trained on combinations of various datasets such as RIMES, Georges Washington, and Los Esposalles. We show that the choice of the training datasets and the merging methods are determinant to the results of the transfer learning task.