Search

Automatically identifying characters in movies has attracted researchers’ interest and led to several significant and interesting applications. However, due to the vast variation in character appearance as well as the weakness and ambiguity of available annotation, it is still a challenging problem. In this paper, we investigate this problem with the supervision of actor-character name correspondence provided by the movie cast. Our proposed framework, namely, Cast2Face, is featured by: 1) we restrict the assigned names within the set of character names in the cast; 2) for each character, by using the corresponding actor and movie name as keywords, we retrieve from the Google image search and get a group of face images to form the gallery set; 3) the probe face tracks in the movie are then identified as one of the actors by a robust kernel multitask joint sparse representation and classification method; and 4) the conditional random field model with consideration of the constraints between face tracks is introduced to enhance the final labeling. Finally, the assigned actor name of a face track is then mapped to the character name based on the cast again. Besides face naming, we further apply the proposed method to spotlight the summarization of a particular actor in his/her movies. We conduct extensive experiments and empirical evaluations on several feature-length movies to demonstrate the satisfying performance of our method. [ABSTRACT FROM AUTHOR]

In this paper, a novel system is developed for synthesizing user-specified emotions onto arbitrary input images or videos. Other than defining the visual affective model based on empirical knowledge, a data-driven learning framework is proposed to extract the emotion-related knowledge from a set of emotion-annotated images. In a divide-and-conquer manner, the images are clustered into several emotion-specific scene subgroups for model learning. The visual affection is modeled with Gaussian mixture models based on color features of local image patches. For the purpose of affective filtering, the feature distribution of the target is aligned to the statistical model constructed from the emotion-specific scene subgroup, through a piecewise linear transformation. The transformation is derived through a learning algorithm, which is developed with the incorporation of a regularization term enforcing spatial smoothness, edge preservation, and temporal smoothness for the derived image or video transformation. Optimization of the objective function is sought via standard nonlinear method. Intensive experimental results and user studies demonstrate that the proposed affective filtering framework can yield effective and natural effects for images and videos. [ABSTRACT FROM PUBLISHER]

In this work, we propose an alternative ground truth to the eye fixation map in visual attention study, called touch saliency. As it can be directly collected from the recorded data of users’ daily browsing behavior on widely used smart phone devices with touch screens, the touch saliency data is easy to obtain. Due to the limited screen size, smart phone users usually move and zoom in the images, and fix the region of interest on the screen when browsing images. Our studies are two-fold. First, we collect and study the characteristics of these touch screen fixation maps (named touch saliency) by comprehensive comparisons with their counterpart, the eye-fixation maps (namely, visual saliency). The comparisons show that the touch saliency is highly correlated with the eye fixations for the same stimuli, which indicates its utility in data collection for visual attention study. Based on the consistency between both touch saliency and visual saliency, our second task is to propose a unified saliency prediction model for both visual and touch saliency detection. This model utilizes middle-level object category features extracted from pre-segmented image superpixels as input to the recently proposed multitask sparsity pursuit (MTSP) framework for saliency prediction. Extensive evaluations show that the proposed middle-level category features can considerably improve the saliency prediction performance when taking both touch saliency and visual saliency as ground truth. [ABSTRACT FROM PUBLISHER]

In this paper, we present an intelligent photography system which can recommend the most user-favored view rectangle for arbitrary camera input, from a photographic compositional perspective. Automating this process is difficult, due to the subjectivity of human's aesthetics judgement and large variations of image contents, where heuristic compositional rules lack generality. Motivated by the recent prevalence of photo-sharing websites, e.g., Flickr.com, we develop a learning-based framework which discovers the underlying aesthetic photographic compositional structures from a large set of user-favored online sharing photographs and utilizes the implicitly shared knowledge among the professional photographers for aesthetically optimal view recommendation. In particular, we propose an Omni-Range Context method which explicitly encodes the spatial and geometric distributions of various visual elements in the photograph as well as cooccurrence characteristics of visual element pairs by using generative mixture models. Searching the optimal view rectangle is then formulated as maximum a posterior by imposing the trained prior distributions along with additional photographic constraints. The proposed system has the potential to operate in near real-time. Comprehensive user studies well demonstrate the effectiveness of the proposed framework for aesthetically optimal view recommendation. [ABSTRACT FROM PUBLISHER]