Search

In this paper, we focus on the task of assigning labels to the over-segmented image patches in a weakly supervised manner, in which the training images contain the labels but do not have the labels' locations in the images. We propose a unified discriminative graph inferring framework by simultaneously inferring patch labels and learning the patch appearance models. On one hand, graph inferring reasons the patch labels by a graph propagation procedure. The graph is constructed by connecting the nearest neighbors which share the same image label, and multiple correlations among patches and image labels are imposed as constraints to the inferring. On the other hand, for each label, the patches which do not contain the target label are adopted as negative samples to learn the appearance model. In this way, the predicted labels will be more accurate in the propagation. Graph inferring and the learned patch appearance models are finally embedded to complement each other in one unified formulation. Experiments on three public datasets demonstrate the effectiveness of our method in comparison with other baselines.

Social media sites (e.g., Flickr) generate a huge amount of landmark photos with temporal information in the real-world, such as the photos describing the events happening near landmarks, and those showing different seasonal sceneries. Analyzing this temporal information of landmarks can benefit various applications, such as landmark timeline construction and tour recommendation. In this paper, we propose a novel Incremental Spatio-Temporal Theme Model (ISTTM), which can incrementally mine temporal themes that characterize the temporal information of landmarks, by differentiating them from the other three kinds of themes, i.e., general themes shared by most of all landmarks, local themes related to certain landmarks and the background theme including non-informative content. ISTTM works in an online way and is capable of selectively processing the updates of the distributions on different types of themes. Based on the proposed ISTTM, we present a framework, namely Temporal Theme Analysis for Landmarks (TTAL), which enables both periodic theme detection from discovered temporal themes and temporal theme visualization by selecting the relevant photos. We have conducted experiments on a large-scale landmark dataset from Flickr. Qualitative and quantitative evaluation results demonstrate the effectiveness of the ISTTM as well as the TTAL framework.

The bag-of-words (BOW) based methods are widely used in image classification. However, huge number of visual information is omitted inevitably in the quantization step of the BOW. Recently, NBNN and its improved methods like Local NBNN were proposed to solve this problem. Nevertheless, these methods do not perform better than the state-of-the-art BOW based methods. In this paper, based on the advantages of BOW and Local NBNN, we introduce a novel locality discriminative coding (LDC) method. We convert each low level local feature, such as SIFT, into code vector using the Local Feature-to-Class distance other than by k-means quantization. After coding, sum-pooling combined with SPM is used to construct a single feature representation vector for each image. Extensive experimental results on several challenging benchmark datasets show that our LDC method outperforms six state-of-the-art image classification methods.

Tracking multiple objects is critical to automatic video content analysis and virtual reality. The major problem is how to solve data association problem when ambiguous measurements are caused by objects in close proximity. To tackle this problem, we propose a multiple information fusion-based multiple hypotheses tracking algorithm integrated with appearance feature, local motion pattern feature and repulsion---inertia model for multi-object tracking. Appearance model based on HSV---local binary patterns histogram and local motion pattern based on optical flow are adopted to describe objects. A likelihood calculation framework is proposed to incorporate the similarities of appearance, dynamic process and local motion pattern. To consider the changes in appearance and motion pattern over time, we make use of an effective template updating strategy for each object. In addition, a repulsion---inertia model is adopted to explore more useful information from ambiguous detections. Experimental results show that the proposed approach generates better trajectories with less missing objects and identity switches.

In this paper, we propose an attention-based virtual content insertion solution, called @ICT. Virtual content insertion (VCI) is an emerging application of video analysis and has been used in video augmentation and advertisement insertion. An ideal VCI solution should make the inserted virtual content being noticed by audiences and at the same time should not interfere with audiences' viewing experience on the original content. To balance these two conflicting issues, meaning high attention and low intrusiveness, we choose higher attentive shots as insertion time while determine insertion place and content interdependently by considering lower attention together with visual consistency. We also propose a measurement of intrusiveness from the viewpoint of visual attention. Furthermore, @ICT includes an in-scene insertion module, which embeds the virtual content into the videos with higher vividness and lower intrusiveness. @ICT is able to obtain an optimal balance between the noticing of the virtual content by audiences and disruption of viewing experience to the original content. It needs little prior knowledge and is applied to general videos. Extensive quantitative and qualitative evaluations on the VCI result have verified the effectiveness of the solution.

This study examines an automatic broadcast soccer video composition system. The research is important as the ability to automatically compose broadcast sports video will not only improve broadcast video generation efficiency, but also provides the possibility to customize sports video broadcasting. We present a novel approach to the two major issues required in the system's implementation, specifically the camera view selection/switching module and the automatic replay generation module. In our implementation, we use multi-modal framework to perform video content analysis, event and event boundary detection from the raw unedited main/sub-camera captures. This framework explores the possible cues using mid-level representations to bridge the gap between low-level features and high-level semantics. The video content analysis results are utilized for camera view selection/switching in the generated video composition, and the event detection results and mid-level representations are used to generate replays which are automatically inserted into the broadcast soccer video. Our experimental results are promising and found to be comparable to those generated by broadcast professionals.

This paper proposes a set of digital watermarking schemes for WAV audio, WAV-table synthesis audio and compressed audio. The watermark embedding scheme is closely related to audio content and based on the human auditory system. The experimental results in listening and robustness illustrate that the proposed watermarking schemes can achieve an optimal balance between audibility and robustness of the watermarked audio. The proposed methods are also very useful and effective for copyright protection, tracing illegal distribution and other applications.

International Conference on Internet Multimedia Computing and Services (ICIMCS) is an annual conference sponsored by ACM SIGMM China Chapter. The conference is especially interested in the latest technologies and applications that deal with the web-scale processing and management of heterogeneous data from the Internet for multimedia computing and service. ICIMCS 2011 held in Chengdu, China— the ancient hometown of lovely panda. The conference has attracted around 80 participants, including researchers from academia and industries across ten countries/regions, for sharing their recent works in the topics ranging from visual information analysis and mining, query processing and search, multimedia privacy and security. This special issue comprises the extended versions of five papers, including two best papers, and three papers from the regular and special sessions of ICIMCS 2011. These papers cover key issues in multimedia computing, including the leveraging of Internet resources for multi-modality fusion and visual classifier learning, preprocessing and indexing of million-scale Internet data, and sharing and streaming of Internet videos. Some of these techniques also demonstrate applications for emerging Internet services, such as video recommendation and product search system, by processing and modeling the heterogeneous forms of resources associated with multimedia data. The first two papers are about the web-scale processing of Internet data. The paper entitled ‘‘Video Recommendation over Multiple Information Sources’’, presented by Meng Wang and his colleagues from National University of Singapore, proposes a unified framework that explores heterogeneous information sources for video recommendation. The framework, based on multi-task SVM learning, aggregates multiple ranked lists generated from personal data, social network, and video metadata into an optimized list for recommendation. The framework is experimented on a large video dataset composed of 1-month social activities happened on Facebook and YouTube websites by 76 users. The second paper entitled ‘‘Multi-label Multiinstance Learning with Missing Object Tags’’, presented by Yi Shen and his colleagues from University of North Carolina at Charlotte, proposes a web-scale learning of object classifiers for free from a collection of user-tagged Flickr images as many as 10 millions. Particularly, the paper addresses three important issues toward fully automatic learning: scalable filtering of spam tags by distributed image clustering; joint modeling of loose tags and missing tags by multiple instance learning that is capable of performing tag prediction; structural learning that takes into account the object relationship to train discriminant classifiers. The next two papers address the search and mining of visual instances. The paper entitled ‘‘Combining Global and Local Matching of Multiple Features for Precise Item Image Retrieval’’, co-authored by Haojie Li and his colleagues from Dalian University of Technology and Nanjing C.-W. Ngo (&) City University of Hong Kong, Hong Kong, Hong Kong e-mail: cscwngo@cityu.edu.hk