Search

Emotional factors directly reflect audiences' attention, evaluation and memory. Recently, video affective content analysis attracts more and more research efforts. Most of the existing methods map low-level affective features directly to emotions by applying machine learning. Compared to human perception process, there is actually a gap between low-level features and high-level human perception of emotion. In order to bridge the gap, we propose a three-level affective content analysis framework by introducing mid-level representation to indicate dialog, audio emotional events (e.g., horror sounds and laughters) and textual concepts (e.g., informative keywords). Mid-level representation is obtained from machine learning on low-level features and used to infer high-level affective content. We further apply the proposed framework and focus on a number of case studies. Audio emotional event, dialog and subtitle are studied to assist affective content detection in different video domains/genres. Multiple modalities are considered for affective analysis, since different modality has its own merit to evoke emotions. Experimental results shows the proposed framework is effective and efficient for affective content analysis. Audio emotional event, dialog and subtitle are promising mid-level representations. © 2012 Springer Science+Business Media, LLC.

© 2013 Springer Science+Business Media, LLC. All rights reserved. In this paper, we proposed a novel and accurate visual-based vehicle type classification system. The system builts up a classifier through applying Support Vector Machine with various features of vehicle image. We made three contributions here: first, we originally incorporated color of license plate in the classification system. Moreover, the vehicle front was measured accurately based on license plate localization and background-subtraction technique. Finally, type probabilities for every vehicle image were derived from eigenvectors rather than deciding vehicle type directly. Instead of calculating eigenvectors from the whole body images of vehicle in existing methods, our eigenvectors are calculated from vehicle front images. These improvements make our system more applicable and accurate. The experiments demonstrated our system performed well with very promising classification rate under different weather or lighting conditions.

Different from the existing work focusing on emotion type detection, the proposed approach in this paper provides flexibility for users to pick up their favorite affective content by choosing either emotion intensity levels or emotion types. Specifically, we propose a hierarchical structure for movie emotions and analyze emotion intensity and emotion type by using arousal and valence related features hierarchically. Firstly, three emotion intensity levels are detected by using fuzzy c-mean clustering on arousal features. Fuzzy clustering provides a mathematical model to represent vagueness, which is close to human perception. Then, valence related features are used to detect five emotion types. Considering video is continuous time series data and the occurrence of a certain emotion is affected by recent emotional history, conditional random fields (CRFs) are used to capture the context information. Outperforming Hidden Markov Model, CRF relaxes the independence assumption for states required by HMM and avoids bias problem. Experimental results show that CRF-based hierarchical method outperforms the one-step method on emotion type detection. User study shows that majority of the viewers prefer to have option of accessing movie content by emotion intensity levels. Majority of the users are satisfied with the proposed emotion detection. © 2012 Elsevier B.V.

While the conversion from mild cognitive impairment to Alzheimer's disease has received much recent attention, the transition from normal cognition to mild cognitive impairment is largely unexplored. The present pattern recognition study addressed this by using neuropsychological test scores and neuroimaging morphological measures to predict the later development of mild cognitive impairment in cognitively normal community-dwelling individuals aged 70-90. years. A feature selection algorithm chose a subset of neuropsychological and FreeSurfer-derived morphometric features that optimally differentiated between individuals who developed mild cognitive impairment and individuals who remained cognitively normal. Support vector machines were used to train classifiers and test prediction performance, which was evaluated via 10-fold cross-validation to reduce variability. Prediction performance was greater when using a combination of neuropsychological scores and morphological measures than when using either of these alone. Results for the combined method were: accuracy 78.51%, sensitivity 73.33%, specificity 79.75%, and an area under the receiver operating characteristic curve of 0.841. Of all the features investigated, memory performance and measures of the prefrontal cortex and parietal lobe were the most discriminative. Our prediction method offers the potential to detect elderly individuals with apparently normal cognition at risk of imminent cognitive decline. Identification at this stage will facilitate the early start of interventions designed to prevent or slow the development of Alzheimer's disease and other dementias. © 2012 Elsevier Inc.

Different conditions, such as occlusions, changes of lighting, shadows and rotations, make vehicle type classification still a challenging task, especially for real-time applications. Most existing methods rely on presumptions on certain conditions, such as lighting conditions and special camera settings. However, these presumptions usually do not work for applications in real world. In this paper, we propose a robust vehicle type classification method based on adaptive multi-class Principal Components Analysis (PCA). We treat car images captured at daytime and night-time separately. Vehicle front is extracted by examining vehicle front width and the location of license plate. Then, after generating eigenvectors to represent extracted vehicle fronts, we propose a PCA method with self-clustering to classify vehicle type. The comparison experiments with the state of art methods and real-time evaluations demonstrate the promising performance of our proposed method. Moreover, as we do not find any public database including sufficient desired images, we built up online our own database including 4924 high-resolution images of vehicle front view for further research on this topic. © 2012 IEEE.

Different conditions, such as occlusions, changes of lighting, shadows and rotations, make vehicle type classification still a challenging task, especially for real-time applications. Most existing methods rely on presumptions on certain conditions, such as lighting conditions and special camera settings. However, these presumptions usually do not work for applications in real world. In this paper, we propose a robust vehicle type classification method based on adaptive multi-class Principal Components Analysis (PCA). We treat car images captured at daytime and night-time separately. Vehicle front is extracted by examining vehicle front width and the location of license plate. Then, after generating eigenvectors to represent extracted vehicle fronts, we propose a PCA method with self-clustering to classify vehicle type. The comparison experiments with the state of art methods and real-time evaluations demonstrate the promising performance of our proposed method. Moreover, as we do not find any public database including sufficient desired images, we built up online our own database including 4924 high-resolution images of vehicle front view for further research on this topic

One of the major objectives in multimedia research is to provide pervasive access and personalized use of multimedia information. Pervasive access of video data implies the access of cognitive and affective aspects of video content. Personalized use requires the services satisfy individual user's needs on video content. This article attempts to provide a content-on-demand (CoD) video adaptation solution by considering users' preference on cognitive content and affective content for video media in general, sports video and movies in particular. In this article, CoD video adaptation system is developed to support users' decision in selecting their content of interest and adaptively deliver video source by selecting relevant content and dropping frames while considering network conditions. First, video contents are annotated by the description schemes (DSs) provided by MPEG-7 multimedia description schemes (MDSs). Then, to achieve a generic adaptation solution, the adaptation is developed following MPEG-21 Digital Item Adaptation (DIA) framework. We study the MPEG-21 reference software on XML generation and develop our own system for CoD video adaptation in three steps: (1) the content information is parsed from MPEG-7 annotation XML file together with bitstream to generate generic Bitstream Syntax Description (gBSD); (2) Users' preference, network characteristic and adaptation QoS (AQoS) are considered for making adaptation decision; (3) adaptation engine automatically parses adaptation decisions and gBSD to achieve adaptation. Unlike most existing adaptation work, the system adapts the content of interest in the video stream according to users' preference. We implement the above-mentioned MPEG-7 and MPEG-21 standards and provide a generic video adaptation solution. Adaptation based on gBSD avoids complex video computation. Thirty students from various departments were invited to assess the system and their responses have been positive. © 2012 Xu et al.

Accurate pedestrian counting are challenging in real-world due to occlusions, pedestrians' overlays or camera view sensitive. In this paper, we propose an accurate and robust pedestrian detection and counting system to address these problems. Our proposed method is group-based, where the count of people in a dense moving group is estimated as a whole. Moving groups containing single or several pedestrians are discriminated from other moving objects. Our method utilizes 9 features of each moving group within a video frame to estimate the pedestrian number in each group. Pedestrian counts are optimized by a novel tracking method, which is based on an analysis of moving groups match, split or merge. Comparison experiments with other two current methods on three benchmark surveillance videos show the effectiveness of our proposed method. © 2012 Springer-Verlag.

Driver assistance system enhances traffic safety and efficiency. Accurate 3D pose of front vehicle can help driver to make right decisions on road. We propose a novel real-time system to estimate 3D pose of the front vehicle. This system consists of two parallel threads: vehicle rear tracking and mapping. Vehicle rear is firstly identified in the video captured by an on-board camera, after license plate localization and foreground extraction. 3D pose estimation technique is then employed with respect to extracted vehicle rear. Most 3D pose estimation techniques need prior models or a stereo initialization with user cooperation. It is extremely difficult to obtain prior models due to various appearances of vehicle rears. Moreover, it is unsafe to ask for driver's cooperation when vehicle is running. In our system, two initial key frames for stereo algorithm are automatically extracted by vehicle rear detection and tracking. Map points are defined as a collection of point features extracted from vehicle rear with their 3D information. These map points are inferences that relating 2D features detected in following vehicle rears with 3D world. Relative 3D Pose between current vehicle rear and on-board camera is then estimated through mapping that matches map points with current point features. We demonstrate the abilities of our system by augmented reality, which needs accurate and real-time 3D pose estimation. © 2012 IEEE.

Many real-world systems (such as cellular telephones and transportation) are acyclic multi-state information networks (AMIN). These networks are composed of multi-state nodes, with different states determined by a set of nodes that receive a signal directly from these multi-state nodes, without satisfying the conservation law. Evaluating the AMIN reliability arises at the design and exploitation stage of many types of technical systems. However, existing analytical methods fail to estimate AMIN reliability in a realistic time frame, even for smaller-sized AMINs. Hence, the main purpose of this article is to present a cellular automata hybrid quasi-Monte Carlo simulation (CA-HMC) by combining cellular automata (CA, to rapidly determine network states), pseudo-random sequences (PRS, to obtain the exibility of the network) and quasi-random sequences (QRS, to improve the accuracy) to obtain a high-quality estimation of AMIN reliability in order to improve the calculation efficiency. We use one benchmark example from well-known algorithms in literature to show the utility and performance of the proposed CA-HMC simulation when evaluating the one-to-all AMIN reliability. © 2012 ISSN 1349-4198.

Gender classification is challenging. Methods for gender classification need to discriminate subtle differences between male and female. Bag-of-Features (BoF) method with sparse coding has been proven very powerful in image classification. In this paper, we apply BoF method for gender classification. We use two sets of images: training images and testing images. All images are represented by a set of Scale-Invariant Feature Transform (SIFT) descriptors. In training stage, using sparse coding, Visual Words Dictionary (VWD) is constructed from SIFT descriptors extracted from training images. In testing, SIFT descriptors of testing images are approximated by visual words in VWD. The choices of approximating visual words determine the classification decision. We apply our method and another two popular methods on public dataset for gender classification. We achieved promising results. Copyright © 2012 ACM.

Driver assistant systems enhance traffic safety and efficiency. The accurate 3D pose of a front vehicle can help a driver to make the right decision on the road. We propose a novel real-time system to estimate the 3D pose of the front vehicle. This system consists of two parallel threads: vehicle rear tracking and mapping. The vehicle rear is first identified in the video captured by an onboard camera, after license plate localization and foreground extraction. The 3D pose estimation technique is then employed with respect to the extracted vehicle rear. Most current 3D pose estimation techniques need prior models or a stereo initialization with user cooperation. It is extremely difficult to obtain prior models due to the varying appearance of vehicles' rears. Moreover, it is unsafe to ask for drivers' cooperation when a vehicle is running. In our system, two initial keyframes for stereo algorithms are automatically extracted by vehicle rear detection and tracking. Map points are defined as a collection of point features extracted from the vehicle's rear with their 3D information. These map points are inferences that relate the 2D features detected in following vehicles' rears with the 3D world. The relative 3D pose of the onboard camera to the front vehicle rear is then estimated through matching the map points with point features detected on the front vehicle rear. We demonstrate the capabilities of our system by testing on real-time and synthesized videos. In order to make the experimental analysis visible, we demonstrated an estimated 3D pose through augmented reality, which needs accurate and real-time 3D pose estimation. © 2011 Peng et al.

Amnestic mild cognitive impairment (aMCI) is a syndrome widely considered to be prodromal Alzheimer's disease. Accurate diagnosis of aMCI would enable earlier treatment, and could thus help minimize the prevalence of Alzheimer's disease. The aim of the present study was to evaluate a magnetic resonance imaging-based automated classification schema for identifying aMCI. This was carried out in a sample of community-dwelling adults aged 70-90. years old: 79 with a clinical diagnosis of aMCI and 204 who were cognitively normal. Our schema was novel in using measures of both spatial atrophy, derived from T1-weighted images, and white matter alterations, assessed with diffusion tensor imaging (DTI) tract-based spatial statistics (TBSS). Subcortical volumetric features were extracted using a FreeSurfer-initialized Large Deformation Diffeomorphic Metric Mapping (FS. +. LDDMM) segmentation approach, and fractional anisotropy (FA) values obtained for white matter regions of interest. Features were ranked by their ability to discriminate between aMCI and normal cognition, and a support vector machine (SVM) selected an optimal feature subset that was used to train SVM classifiers. As evaluated via 10-fold cross-validation, the classification performance characteristics achieved by our schema were: accuracy, 71.09%; sensitivity, 51.96%; specificity, 78.40%; and area under the curve, 0.7003. Additionally, we identified numerous socio-demographic, lifestyle, health and other factors potentially implicated in the misclassification of individuals by our schema and those previously used by others. Given its high level of performance, our classification schema could facilitate the early detection of aMCI in community-dwelling elderly adults. © 2011 Elsevier Inc.

AdaBoost classifiers with Haar-like features are widely used for license plate (LP) localization. However, it normally requires high-dimensional Haar-like features which cause extremely high computational cost. In this paper, a rejection cascade was built for LP localization with reduced Haar-like features. We first introduced line segment features as pre-input of Haarlike features for AdaBoost to eliminate more than 70% of the background in an image. Line segment features, including density, directionality and regularity, were extracted from line segments, which were detected by applying Hough Transform on an edge image. Later, AdaBoost classifiers with Haar-like features were further applied to identify the exact location of license plates. Our method dramatically reduced the demanded dimensions of Haar-like features, therefore saved much time in AdaBoost training stage. By comparing our method with methods of only using Haar-like features and only using line segment features, experimental results demonstrated that our proposed method achieved the best detection rate with significantly reduced dimensions of Haar-like features. © 2011 IEEE.

Movie shot classification is vital but challenging task due to various movie genres, different movie shooting techniques and much more shot types than other video domain. Variety of shot types are used in movies in order to attract audiences attention and enhance their watching experience. In this paper, we introduce context saliency to measure visual attention distributed in keyframes for movie shot classification. Different from traditional saliency maps, context saliency map is generated by removing redundancy from contrast saliency and incorporating geometry constrains. Context saliency is later combined with color and texture features to generate feature vectors. Support Vector Machine (SVM) is used to classify keyframes into pre-defined shot classes. Different from the existing works of either performing in a certain movie genre or classifying movie shot into limited directing semantic classes, the proposed method has three unique features: 1) context saliency significantly improves movie shot classification; 2) our method works for all movie genres; 3) our method deals with the most common types of video shots in movies. The experimental results indicate that the proposed method is effective and efficient for movie shot classification. © 2011 IEEE.

Close-up (CU) is a photographic technique which tightly frames a person or an object. In movies, it is applied to guide audience attention and to evoke audience emotion. In this paper, we detect face CU, object CU, and lean of movies, which are widely used to romance emotions. A lean consists of shots in a sequence, with a close-up shot as focus. A set of features are extracted by considering movie making techniques and human attention for CU detection. The features are average saliency, color entropy, color variance, face height, skin area, and texture scales. These features are tested through statistical hypothesis test to be significantly discriminating for CUs. Then, Support Vector Machine (SVM) is applied on these features to detect face CU and object CU. Based on the face CU and object CU detection result, lean is further detected by investigating the changing of the face/object size. Lean detection is of challenge due to the technique of montage. We solve this problem through color similarity estimation and SIFT point matching. Experimental results on four full length movies verify the effectiveness of the proposed method. © 2010 IEEE.

Movie affective content analysis attracts increasing research efforts since affective content not only affect users attentions but also locate movie highlights. However, affective content retrieval is still a challenging task due to the limitation of affective features in movies. Scripts provide direct access to the movie content and represent affective aspects of the movie. In this paper, we utilize scripts as an important clue to retrieve video affective content. The proposed approach includes two main steps. Firstly, affective script partitions are extracted by detecting emotional words. Secondly, affective partitions are validated by using visual and auditory features. The results are encouraging and compared with the manually labelled ground truth. © 2010 Springer-Verlag.

Cervical cancer is the second most common malignancy among women worldwide, if it is detected in early stage, cure rate is relatively high. Computer aided abnormality detection for cervical smear is developed to assist medical experts to handle the microscopy images, examine cell abnormalities and diagnose dyskaryosis. The microscopy images of cells in cervix uteri are stained by the tumor marker Ki-67, so that the abnormal nuclei present brown while normal ones are bluish. Segmentation is the most important and difficult task to calculate the ratio of abnormal nuclei to all nuclei. In order to achieve accurate segmentation of nuclei, we propose a multi-level segmentation approach for abnormality identification in microscopy images. First level segmentation aims to partition abnormal (stained) nuclei regions and all nuclei regions. Because of under-segmentation after first level segmentation, second level segmentation is applied to further partition the clustered nuclei. In order to classify touching regions of clustered nuclei and separate regions of single nucleus, relevant meaningful features are extracted from regions of interest. Consequently all the nuclei regions are separated and in conjunction with the abnormal nuclei regions in the first level segmentation, the abnormality i.e. ratio of abnormal nuclei to all nuclei is obtained. Experimental results indicate that our method achieved an accuracy of 93.55% and 95.8% in term of abnormal nuclei and all nuclei respectively for identification of abnormalities. Our proposed method produces a satisfactory segmentation. © 2010 IEEE.

Video becomes a crucial information resource in last decades, because of the rapid development of camera as well as the internet explosion. High-quality video sequences are always desired in lots of fields. Since the bottleneck of data storage and interferences of shooting condition, we cannot always obtain high-resolution video. This botheration can be circumvented by super-resolution. Currently, almost super-resolution techniques are in the framework of Maximum a Posterior (MAP). Appropriate parameters of prior distribution are crucial for recovering accurate super-resolution image. We utilise a novel Weighted Cross Validation (WCG) method to learn theses prior parameters. Comparison experiments are provided to illustrate the effectiveness of our approach. Copyright 2010 ACM.

This paper presents an efficient and innovative method for the automated counting of cells in a microscopic image. The performance of watershed-based algorithms for the segmentation of clustered cells has been well demonstrated. The strength of our algorithm lies in the fact that it incorporates knowledge of color in the image. Our method uses the watershed transform with iterative shape alignment and is shown to be more accurate in retaining cell shape. We report a sensitivity of 97% and specificity of 96% when all color bands are used. Our methods could be of value to computer-based systems designed to objectively interpret microscopic images, since they provide a means for accurate cell segmentation. © 2010 IEEE.

Cervical cancer is the second most common cancer among women. Meanwhile, cervical cancer could be largely preventable and curable with regular Pap tests. Nuclei changes in the cervix could be found by this test. Accurate nuclei detection is extremely critical as it is the previous step of analysing nuclei changes and diagnosis afterwards. Recently, computer-aided nuclei segmentation has increased dramatically. Although such algorithms could be utilised in the situation for sparse nuclei since they are intuitively detected, the segmentation for the complicated nuclei clusters is still challenging task. This paper presents a new methodology for the detection of cervical nuclei clusters. We first detect all the nuclei from the cervical microscopic image by an ellipse fitting algorithm. Second, we chose some high-relevant features from all the features we obtained in last step via F-score, which is based on to what extent one feature attributes to results. All the ellipses are then classified into single ones and cluster ones by C4.5 decision tree with selected features. We evaluated the performance of this method by the classification accuracy, sensitivity, and cluster predictive value. With the 9 selected features from the original 13 features, we came by the promising classification accuracy (97.8%). © 2010 IEEE.

Affective analysis attracts increasing attention in multimedia domain since affective factors directly reflect audiences' attention, evaluation and memory. Existing study focuses on mapping low-level affective features to high-level emotions by applying machine learning methods. Therefore, choosing effective features and developing efficient machine learning algorithms become vital for affective analysis. In this paper, we investigate the effectiveness of a novel classification approach, called Adaptive Local Hyperplanes (ALH), in affective analysis. The reason ALH is appealing in affective analysis is two-fold. Firstly, affective features are not equally important for emotion categories; ALH inherently assigns feature weights based on discriminative ability of each feature. Secondly, ALH achieves competitive performance with state-of-the-art classifiers (e.g., SVM) while it is designed for multi-class classification. Consequently, it is worthwhile to explore the usage of ALH in affective analysis. MTV data are used in this study. As the first effort of applying ALH to affective analysis, the results presented in this paper provide a foundation for future research in affective analysis. Copyright 2010 ACM.

We propose a novel framework that facilitates automatic editing and authoring of multimedia using static and moving cameras in a dynamic scene. The framework incorporates several video techniques such as object tracking using mean shift and object recognition using Scaled Invariant Feature Transform (SIFT). These techniques are linked together by a comprehensive yet simple-to-program script authoring mechanism based on video event detection. These combined features empower the system to play a virtual director role in live video stream editing and multimedia integration. The system requires minimum human intervention and can leverage production efficiency for both novice and professional users. The experimental results from our prototype system demonstrate that this framework is achievable using inexpensive hardware and standard video cameras. Our system provides comprehensive pre-production authoring capabilities that lend towards integration of video and heterogonous multimedia elements in realtime. We have found this framework to be useful in many applications such as live video streaming, distance education, live entertainment, sports coverage and personal video broadcasting. © 2005 IEEE.

We use SIMD instructions to implement a popular video object-tracking algorithm in an attempt to achieve the best possible performance on the available hardware. We start with an implementation of the well-known mean shift algorithm with adaptive scale and background-weighted histogram enhancements. Due to its histogramic nature, mean shift is not normally considered worth optimizing with vector instructions. However, when we looked for basic opportunities to exploit SIMD style processing, we in fact discovered several opportunities to exploit SIMD instructions. We compare the tracking efficiency of our implementation with a standard implementation in order to demonstrate the efficiency of the modified technique. In this paper we present the optimizations we have applied as well as the performance results obtained. © 2005 IEEE.

In recent times, there are increasing numbers of computer vision and pattern recognition (CVPR) technologies being applied to real time video processing using single processor PCs. However, these multiple computational expensive tasks are generating bottlenecks in real-time processing. We propose a scheme to achieve both high throughput and accommodation to user-specified scheduling rules. The scheduler is then distributing 'slices' of the latency insensitive tasks such as video object recognition and facial localization among the latency sensitive ones. We show our proposed work in detail, and illustrating its application in a real-time e-learning streaming system. We also provide discussions into the scheduling implementations, where a novel concept using interleaved SIMD execution is discussed. The experiments have indicated successful scheduling results on a high end consumer grade PC. © Springer-Verlag Berlin Heidelberg 2005.

We present a dependence testing algorithm that considers the short width of modern SIMD registers in a typical microprocessor. The test works by solving the dependence system with the generalized GCD algorithm and then simplifying the solution equations for a particular set of dependence distances. We start by simplifying each solution lattice to generate points that satisfy some small constant dependence distance that corresponds to the width of the register being used. We use the Power Test to efficiently perform Fourier-Motzkin Variable Elimination on the simplified systems in order to determine if dependences exist. The improvements described in this paper also extend our SIMD dependence test to loops with symbolic and triangular lower and upper bounds as well as array indices that contain unknown symbolic additive constants. The resulting analysis is used to guide the vectorization pass of a dynamic multimedia compiler used to compile software agents that process audio, video and image data. We fully detail the proposed dependence test in this paper, including the related work.

We propose a novel framework that employs correlated digital video and audio watermarking where the watermarking sequence contains video interaction information and media features as a basis towards constructing a secure, self-contained, format independent stream-based scheme for interactive media authoring. We have detailed our methods and illustrate the advantages of this scheme over the traditional meta-tag approach for interactive video authoring. We also identify many of the potential applications that can benefit from this approach. We outline the prototype application and demonstrate the successful implementation within our existing real-time multimedia authoring system, where the new post production video delivering module is based on this framework. © 2005 IEEE.

We propose a novel adaptive approach for object tracking using fast colour thresholding and region merging. It proves to be an effective measure against large variations between consecutive frames during a mean-shift process. The approach retains mean-shift's property of efficiency and improves mean-shift's drawback of robustness. It can track non-rigid objects with significant occlusion.