Search

In this paper, we present a new video database: CVD2014-Camera Video Database. In contrast to previous video databases, this database uses real cameras rather than introducing distortions via post-processing, which results in a complex distortion space in regard to the video acquisition process. CVD2014 contains a total of 234 videos that are recorded using 78 different cameras. Moreover, this database contains the observer-specific quality evaluation scores rather than only providing mean opinion scores. We have also collected open-ended quality descriptions that are provided by the observers. These descriptions were used to define the quality dimensions for the videos in CVD2014. The dimensions included sharpness, graininess, color balance, darkness, and jerkiness. At the end of this paper, a performance study of image and video quality algorithms for predicting the subjective video quality is reported. For this performance study, we proposed a new performance measure that accounts for observer variance. The performance study revealed that there is room for improvement regarding the video quality assessment algorithms. The CVD2014 video database has been made publicly available for the research community. All video sequences and corresponding subjective ratings can be obtained from the CVD2014 project page (http://www.helsinki.fi/psychology/groups/visualcognition/).

This paper presents a new database, CID2013, to address the issue of using no-reference (NR) image quality assessment algorithms on images with multiple distortions. Current NR algorithms struggle to handle images with many concurrent distortion types, such as real photographic images captured by different digital cameras. The database consists of six image sets; on average, 30 subjects have evaluated 12-14 devices depicting eight different scenes for a total of 79 different cameras, 480 images, and 188 subjects (67% female). The subjective evaluation method was a hybrid absolute category rating-pair comparison developed for the study and presented in this paper. This method utilizes a slideshow of all images within a scene to allow the test images to work as references to each other. In addition to mean opinion score value, the images are also rated using sharpness, graininess, lightness, and color saturation scales. The CID2013 database contains images used in the experiments with the full subjective data plus extensive background information from the subjects. The database is made freely available for the research community.

The affective content of a movie is often considered to be largely determined by its style and aesthetics. Recently, studies have attempted to estimate affective movie content with computational features, but results have been mixed, one of the main reasons being a lack of data on perceptual stylistic and aesthetic attributes of film, which would provide a ground truth for the features. The distinctions between energetic and tense arousal as well as perceived and felt affect are also often neglected. In this study, we present a data set of ratings by 73 viewers of 83 stylistic, aesthetic, and affective attributes for a selection of movie clips containing complete scenes taken from mainstream movies. The affective attributes include the temporal progression of perceived and felt valence and arousal within the clips. The data set is aimed to be used to train algorithms that predict viewer assessments based on low-level computational features. With this data set, we performed a baseline study modeling the relation between a large selection of low-level computational features (i.e., visual, auditory, and temporal) and perceptual stylistic, aesthetic, and affective attributes of movie clips. Two algorithms were compared in a realistic prediction scenario: linear regression and the neural-network-based Extreme Learning Machine (ELM). Felt and perceived affect as well as stylistic attributes were shown to be equally easy to predict, whereas the prediction of aesthetic attributes failed. The performance of the ELM predictor was overall found to be slightly better than the linear regression. A feature selection experiment illustrated that features from all low-level computational modalities, visual, auditory and temporal, contribute to the prediction of the affect assessments. We have made our assessment data and extracted computational features publicly available.

Significant depth judgment errors are common in augmented reality. This study presents a visualization approach for improving relative depth judgments in augmented reality. The approach uses auxiliary augmented objects in addition to the main augmentation to support ordinal and interval depth judgment tasks. The auxiliary augmentations are positioned spatially near real-world objects, and the location of the main augmentation can be deduced based on the relative depth cues between the augmented objects. In the experimental part, the visualization approach was tested in the “X-ray” visualization case with a video see-through system. Two relative depth cues, in addition to motion parallax, were used between graphical objects: relative size and binocular disparity. The results show that the presence of auxiliary objects significantly reduced errors in depth judgment. Errors in judging the ordinal location with respect to a wall (front, at, or behind) and judging depth intervals were reduced. In addition to reduced errors, the presence of auxiliary augmentation increased the confidence in depth judgments, and it was subjectively preferred. The visualization approach did not have an effect on the viewing time.

Television viewers' attention is increasingly more often divided between television and "second screens", for example when viewing television broadcasts and following their related social media discussion on a tablet computer. The attentional costs of such multitasking may vary depending on the ebb and flow of the social media channel, such as its emotional contents. In the present study, we tested the hypothesis that negative social media messages would draw more attention than similar positive messages. Specifically, news broadcasts were presented in isolation and with simultaneous positive or negative Twitter messages on a tablet to 38 participants in a controlled experiment. Recognition memory, gaze tracking, cardiac responses, and self-reports were used as attentional indices. The presence of any tweets on the tablet decreased attention to the news broadcasts. As expected, negative tweets drew longer viewing times and elicited more attention to themselves than positive tweets. Negative tweets did not, however, decrease attention to the news broadcasts. Taken together, the present results demonstrate a negativity bias exists for social media messages in media multitasking; however, this effect does not amplify the overall detrimental effects of media multitasking.

The development of visual retrieval methods requires information about user interaction with images, including their description and categorization. This article presents the development of a categorization model for magazine images based on two user studies. In Study 1, we elicited 10 main classes of magazine image categorization criteria through sorting tasks with nonexpert and expert users (N=30). Multivariate methods, namely, multidimensional scaling and hierarchical clustering, were used to analyze similarity data. Content analysis of category names gave rise to classes that were synthesized into a categorization framework. The framework was evaluated in Study 2 by experts (N=24) who categorized another set of images consistent with the framework and found it to be useful in the task. Based on the evaluation study the framework was solidified into a model for categorizing magazine imagery. Connections between classes were analyzed both from the original sorting data and from the evaluation study and included into the final model. The model is a practical categorization tool that may be used in workplaces, such as magazine editorial offices. It may also serve to guide the development of computational methods for image understanding, selection of concepts for automatic detection, and approaches to support browsing and exploratory image search. © 2011 Wiley Periodicals, Inc.

This poster reports a practitioner evaluation of a multifaceted image categorization model designed for journalistic imagery. We show that the model led to consistent categorizations and was evaluated as useful by the participants in the user test.

Most spatially interlacing stereoscopic 3D displays display odd and even rows of an image to either the left or the right eye of the viewer. The visual system then fuses the interlaced image into a single percept. This row-based interlacing creates a small vertical disparity between the images; however, interlacing may also induce horizontal disparities, thus generating depth artifacts. Whether people perceive the depth artifacts and, if so, what is the magnitude of the artifacts are unknown. In this study, we hypothesized and tested if people perceive interlaced edges on different depth levels. We tested oblique edge orientations ranging from 2 degrees to 32 degrees and pixel sizes ranging from 16 to 79 arcsec of visual angle in a depth probe experiment. Five participants viewed the visual stimuli through a stereoscope under three viewing conditions: noninterlaced, interlaced, and row averaged (i.e., where even and odd rows are averaged). Our results indicated that people perceive depth artifacts when viewing interlaced stereoscopic images and that these depth artifacts increase with pixel size and decrease with edge orientation angle. A pixel size of 32 arcsec of visual angle still evoked depth percepts, whereas 16 arcsec did not. Row-averaging images effectively eliminated these depth artifacts. These findings have implications for display design, content production, image quality studies, and stereoscopic games and software.

In recent years, studies have estimated affective movie content computationally with stylistic features, yet knowledge of the perceptual relation between style and affect remains scarce. Such knowledge would be useful in affect-based movie recommendation systems. To this end, a user study was conducted in which seventy-three participants with varying levels of film expertise rated movie clips according to 13 stylistic features in three modalities (visual, aural and temporal) as well as perceived and felt affect in three dimensions (hedonic tone, energetic arousal and tense arousal). Style-based linear regression models were then constructed for each affect dimension. Visual features contributed the most to hedonic tone and tense arousal, and temporal features to energetic arousal. Also, perceived affect showed greater inter-rater agreement and better modeling performance than felt affect. The results indicate that the influence of specific stylistic features on affect varies by dimension and by whether the affect is perceived or felt.

In this work, we study local feature extraction methods and evaluate their performance in detecting local features from the salient regions of images. In order to measure the detectors’ performance, we compared the detected regions to gaze fixations obtained from the eye movement recordings of human participants viewing two types of images: natural images (photographs) and abstract/surreal images. The results indicate that all of the six evaluated local feature detectors perform clearly above chance level. The Hessian-Affine detector performs the best and almost reaches the performance level of state-of-the-art saliency detection methods.

We present a system for automating magazine layout process and data on its performance in a user evaluation test. The purpose of the study is to find the feasibility of the key system variables on the end-user. This semi-automatic system is based on content-based image feature algorithms to automate the layout process. The image related automation includes image cropping, overlaying text on top of images, image color palette creation, and image alignment. The algorithms rely on principles of photography and are used here together in the context of graphic design. For example the rule-of-space and leading line concepts can be extended to layout contexts for improved alignment of text and images. The computation is based on automatic analysis, such as face detection, color saliency, and textureness. We used the automation to create a functional prototype, an iPad magazine, using an open HTML5 eBook framework, which relies on CSS3 media queries for the layout adaptation. User experiments (40 participants) were conducted where the system was compared against two commercial iPad magazine systems in terms of overall usability with emphasis on visual aspects. Questionnaires and free commenting was used in a task-based usage scenario. The experiments were video recorded and the user comments were transcribed and coded into attributes. Usability results mixed with qualitative observation are reported. Results show that usability, read- ability and visuality are important to the users. Our system was considered more usable than the other two systems with some of the defining aspects being simplicity in terms of usability, subjective readability and visual clarity.

Image quality is a vital criterion that guides the technical development of digital cameras. Traditionally, the image quality of digital cameras has been measured using test-targets and/or subjective tests. Subjective tests should be performed using natural images. It is difficult to establish the relationship between the results of artificial test targets and subjective data, however, because of the different test image types. We propose a framework for objective image quality metrics applied to natural images captured by digital cameras. The framework uses reference images captured by a high-quality reference camera to find image areas with appropriate structural energy for the quality attribute. In this study, the framework was set to measure sharpness. Based on the results, the mean performance for predicting subjective sharpness was clearly higher than that of the state-of-the-art algorithm and test-target sharpness metrics.

In this study the perception of crowds was investigated in urban environment. The images of crowds were viewed non-stereoscopically and stereoscopically with HMD (head-mounted display). The task of the participants was to count the number of persons in the crowds. The results clearly indicate that stereoscopic viewing enhances perception of crowds. The counting task was determined to be easiest with stereoscopic viewing, its error rate was significantly smaller and it was significantly preferred to non-stereoscopic viewing. The viewing method did not differ statistically with respect to the completion time.

The added value of stereoscopy is an important factor for stereoscopic product development and content production. Previous studies have shown that 'image quality' does not encompass the added value of stereoscopy, and thus the attributes naturalness and viewing experience have been used to evaluate stereoscopic content. The objective of this study was to explore what the added value of stereoscopy may consist of and what are the content properties that contribute to the magnitude of the added value. The hypothesis was that interestingness is a significant component of the added value. A subjective study was conducted where the participants evaluated three attributes of the stimuli in the consumer photography domain: viewing experience, naturalness of depth and interestingness. In addition to the no-reference direct scaling method a novel method, the recalled attention map, was introduced and used to study attention in stereoscopic images. In the second part of our study, we use eye tracking to compare the salient regions in monoscopic and stereoscopic conditions. We conclude from the subjective results that viewing experience and naturalness of depth do not cover the entire added value of stereoscopy, and that interestingness brings a new dimension into the added value research. The eye tracking data analysis revealed that the fixation maps are more consistent between participants in stereoscopic viewing than in monoscopic viewing and from this we conclude that stereoscopic imagery is more effective in directing the viewer's attention.

Face detection techniques are used for many different applications. For example, face detection is a basic component in many consumer still and video cameras. In this study, we compare the performance of face area data and freely selected local area data for predicting the sharpness of photographs. The local values were collected systematically from images, and for the analyses we selected only the values with the highest performance. The objective sharpness metric was based on the statistics of the wavelet coefficients for the selected areas. We used three image contents whose subjective sharpness values had been measured. The image contents were captured by 13 cameras, and the images were evaluated by 25 subjects. The quality of the cameras ranged from low-end mobile phone cameras to low-end compact cameras. The image contents simulated typical photos that consumers take with their mobile phones. The face area sizes on the images were approximately 0.4, 1.0 or 4.0 %. Based on the results, the face area data proved to be valuable for measuring the sharpness of the photographs if the face size was large enough. When the face area size was 1.0 or 4.0 %, the performance of the measured sharpness values was equal to or better than the sharpness values measured from the best local areas. When the face area was too small (0.4 %), the performance was low compared with the best local areas.

Objective image quality metrics can be based on test targets or algorithms. Traditionally, the image quality of digital cameras has been measured using test targets. Test-target measurements are tedious and require a controlled laboratory environment. Algorithm metrics can be divided into three groups: full-reference (FR), reduced-reference (RR) and noreference (NR). FR metrics cannot be applied to the computation of image quality captured by digital cameras because pixel-wise reference images are missing. NR metrics are applicable only when the distortion type is known and the distortion space is low-dimensional. RR metrics provide a tradeoff between NR and FR metrics. An RR metric does not require a pixel-wise reference image; it only requires a set of extracted features. With the aid of RR features, it is possible to avoid problems related to NR metrics. In this study, we evaluate the applicability of RR metrics to measuring the image quality of natural images captured by digital cameras. We propose a method in which reference images are captured using a reference camera. The reference images represented natural reproductions of the views under study. We tested our method using three RR metrics proposed in the literature. The results suggest that the proposed method is promising for measuring the quality of natural images captured by digital cameras for the purpose of camera benchmarking.

We present a method to evaluate stereo camera depth accuracy in human centered applications. It enables the comparison between stereo camera depth resolution and human depth resolution. Our method uses a multilevel test target which can be easily assembled and used in various studies. Binocular disparity enables humans to perceive relative depths accurately, making a multilevel test target applicable for evaluating the stereo camera depth accuracy when the accuracy requirements come from stereoscopic vision. The method for measuring stereo camera depth accuracy was validated with a stereo camera built of two SLRs (singlelens reflex). The depth resolution of the SLRs was better than normal stereo acuity at all measured distances ranging from 0.7 m to 5.8 m. The method was used to evaluate the accuracy of a lower quality stereo camera. Two parameters, focal length and baseline, were varied. Focal length had a larger effect on stereo camera's depth accuracy than baseline. The tests showed that normal stereo acuity was achieved only using a tele lens. However, a user's depth resolution in a video see-through system differs from direct naked eye viewing. The same test target was used to evaluate this by mixing the levels of the test target randomly and asking users to sort the levels according to their depth. The comparison between stereo camera depth resolution and perceived depth resolution was done by calculating maximum erroneous classification of levels.

The goal of the study was to develop a method for quality computation of digitally printed images. We wanted to use only the attributes which have a meaning for subjective visual quality experience of digitally printed images. Based on the subjective data and our assessments the attributes for quality calculation were sharpness, graininess and color contrast. The proposed graininess metric divides the fine detail image into blocks and used the low energy blocks for graininess calculation. The proposed color contrast metric computes the contrast of dominant colors using the coarse scale image. The proposed sharpness metric divides the coarse scale image into blocks and uses the high energy blocks for sharpness calculation. The reduced reference features of sharpness and graininess metrics are the numbers of high or low energy blocks. The reduced reference features of the color contrast metric are the directions of the dominant colors in reference image. The overall image quality was calculated by combining the values. The performance of proposed application specific image quality metric was high compared to the state of the art reduced reference applicationindependent image quality metric. Linear correlation coefficients between subjective and predicted MOS were 0.88 for electrophotography and 0.98 for ink-jet printed samples, for a sample set of 21 prints for electrophotography and for inkjet, subjectively evaluated by 28 observers.