Search

In this paper we propose the use of enhanced mid-level information, such as information obtained from the application of supervised or unsupervised learning methodologies on low-level characteristics, in order to improve semantic multimedia analysis. High-level, a priori contextual knowledge about the semantic meaning of objects and their low-level visual descriptions are combined in an integrated approach that handles in a uniform way the gap between semantics and low-level features. Prior work on low-level feature extraction is extended and a region thesaurus containing all mid-level features is constructed using a hierarchical clustering method. A model vector that contains the distances from each mid-level element is formed and a neural network-based detector is trained for each semantic concept. Contextual adaptation improves the quality of the produced results, by utilizing fuzzy algebra, fuzzy sets and relations. The novelty of the presented work is the context- driven mid-level manipulation of region types, utilizing a domain-independent ontology infrastructure to handle the knowledge. Early experimental results are presented using data derived from the beach domain.

In this paper previous work on the detection of high-level concepts within multimedia documents is extended by introducing a mid-level ontology as a means of exploiting the visual context of images in terms of the regions they consist of. More specifically, we construct a mid-level ontology, define its relations and integrate it in our knowledge modelling approach. In the past we have developed algorithms to address computationally efficient handling of visual context and extraction of mid-level characteristics and now we explain how these diverse algorithms and methodologies can be combined in order to approach a greater goal, that of semantic multimedia analysis. Early experimental results are presented using data derived from the beach domain.

The semantic gap is the main problem of content based multimedia retrieval. This refers to the extraction of the semantic content of multimedia documents, the understanding of user information needs and requests, as well as to the matching between the two. In this chapter we focus on the analysis of multimedia documents for the extraction of their semantic content. Our approach is based on fuzzy algebra, as well as fuzzy ontological information. We start by outlining the methodologies that may lead to the creation of a semantic index; these methodologies are integrated in a video annotating environment. Based on the semantic index, we then explain how multimedia content may be analyzed for the extraction of semantic information in the form of thematic categorization. The latter relies on stored knowledge and a fuzzy hierarchical clustering algorithm that uses a similarity measure that is based on the notion of context.

In this paper we propose an algorithm to improve the results of knowledge-assisted image analysis, based on contextual information. In order to achieve this, we utilize fuzzy algebra, fuzzy sets and relations, towards efficient manipulation of image region concepts. We provide a novel context modelling, based on the OWL language, using RDF reification. Initial image analysis results are enhanced by the utilization of domain-independent, semantic knowledge in terms of concepts and relations between them. The novelty of the presented work is the context-driven re-adjustment of the degrees of confidence of the detected concepts produced by any image analysis technique, utilizing a domain-independent ontology infrastructure to handle the knowledge, as well as multiple application domains.

Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Vallet, D., Castells, P., Fernandez, M., Mylonas, P., Avrithis, Y. "Personalized Content Retrieval in Context Using Ontological Knowledge". IEEE Transactions on Circuits and Systems for Video Technology, vol. 17, no. 3, pp. 336-346, March 2007, Personalized content retrieval aims at improving the retrieval process by taking into account the particular interests of individual users. However, not all user preferences are relevant in all situations. It is well known that human preferences are complex, multiple, heterogeneous, changing, even contradictory, and should be understood in context with the user goals and tasks at hand. In this paper, we propose a method to build a dynamic representation of the semantic context of ongoing retrieval tasks, which is used to activate different subsets of user interests at runtime, in a way that out-of-context preferences are discarded. Our approach is based on an ontology-driven representation of the domain of discourse, providing enriched descriptions of the semantics involved in retrieval actions and preferences, and enabling the definition of effective means to relate preferences and context., This research was supported by the European Commission (FP6-001765 – aceMedia), and the Spanish Ministry of Science and Education (TIN2005-06885)

In this paper, we extend on previous work in order to address the problem of multimedia personalization at a semantic level. In different previous works, we have developed algorithms to address computationally efficient handling of large but sparse fuzzy relations, and theory to address knowledge representation, thematic categorization and user modeling. In this work we take two further steps: i) we integrate ontologies in our original knowledge modeling approach and ii) we explain how these diverse algorithms and methodologies can be combined in order to approach a greater goal, that of semantic multimedia personalization

In this paper we present a framework for simultaneous image segmentation and region labeling leading to automatic image annotation. The proposed framework operates at semantic level using possible semantic labels to make decisions on handling image regions instead of visual features used traditionally. In order to stress its independence of a specific image segmentation approach we applied our idea on two region growing algorithms, i.e. watershed and recursive shortest spanning tree. Additionally we exploit the notion of visual context by employing fuzzy algebra and ontological taxonomic knowledge representation, incorporating in this way global information and improving region interpretation. In this process, semantic region growing labeling results are being re-adjusted appropriately, utilizing contextual knowledge in the form of domain-specific semantic concepts and relations. The performance of the overall methodology is demonstrated on a real-life still image dataset from the popular domains of beach holidays and motorsports.

One of the most interesting topics in modern multimedia research, as well as one of the most important tasks in modern audiovisual content providing systems is the treatment of content and users at a semantic level. In this framework, mapping user profiles to multimedia content is a challenging and important problem, as the new generation of home television viewers is currently being confronted with a series of technological developments and improvements, targeted towards their expectations from TV broadcasts. This paper presents initial results from our ongoing work in the field of semantic multimedia analysis, retrieval and user profile extraction in the framework of an interactive video environment, within the MELISA project (E. Papaioannou et al., 2004); this project aims at the cross-media broadcasting of sports events featuring interactive advertising and sports-related games over digital television and provides services for personalized presentation of interactive time video content. Initially, it extends on previous work on low level multimedia content, like scene and shot detection, contour extraction and object tracking, descriptor extraction and matching, and semantic document analysis, in the direction of extraction of semantic user preferences. Such preferences can then be utilized towards the personalization of the overall retrieval process and the multimedia content offering to the end-users. To tackle the latter issue, we propose a methodology based mainly on the utilization of a novel mechanism of weights definition, which are combined with the automatically extracted profiling information.

The advent and wide proliferation of Social Web in the recent years has promoted the concept of social interaction as an important influencing factor of the way enterprises and organizations conduct business. Among the fields influenced is that of Enterprise Knowledge Management, where the increase and maintenance of the employees' active participation in the organization's knowledge management activities is pursued through the adoption of social computing approaches. In this paper we consider a prominent and increasingly applied such approach, namely enterprise microblogging, and we propose a novel way to exploit its knowledge generation and sharing capabilities in order to effectively capture and formalize enterprise knowledge that is vague. Application and empirical evaluation of the framework indicates significant potential towards this goal.

It is a common fact that modern e-learning schemes lack educational content representation and user personalisation. In this framework, automated extraction of user profiles, to be used in an e-learning content offering system, forms an interesting and important problem. In this approach we present the design and implementation of such a profile-based system, by which content is matched to its environmental context, so that it can be adapted to its user's needs and capabilities. Current effort extends previous work on profile extraction via clustering techniques and on integrated e-learning systems. It relies on the fundamental IEEE e-learning model, suitably adapted to reflect and focus on profiling aspects of the system.

In this work we propose a saliency based psycho-visual weighting of the BoVW for object recognition. This approach is designed to identify objects related to IADL on videos recorded by a wearable camera. These recording give an egocentric point-of-view on the upcoming action. This point- of-view is also characterized by a complex visual scene with several objects on the frame plan. The human visual system functions is a way to process only the relevant data by considering areas of interest. Based on this idea, we propose a new approach by introducing saliency models to discard irrelevant information in the video frames. Therefore we apply a visual saliency model to weight the image signature within the BoVW framework. Visual saliency is well suited for catching spatio-temporal information related to the observer's attention on the video frame. We also proposed an additional geometric saliency cue that models the anticipation phenomenon observed on subjects watching video content from the wearable camera. The findings show that discarding irrelevant features gives better performances when compared to the baseline method which consider the whole set of features in the images.

Volume 2; With the proliferation of execution contexts (PDAs, mobile phones, blind-friendly interfaces, etc.), medias must be adapted according to the target context. In order to ensure universal access to information, we propose an architecture which uses information retrieval techniques in order to replace incompatible media items with compatible ones. More specifically, we advertise the use of the Web as a repository of potential alternative media items. Moreover, the chosen media item must convey the same message as the original ones, while satisfying the target profile. We show that Web-based search engines can already achieve this to a limited extent. Nonetheless, some results are unsatisfactory because media annotations lack semantics, are partial and are heterogeneous. Hence, we demonstrate the value of Semantic Web technologies, such as RDF descriptions, ontologies, ontology merging and matching, for improving this adaptation framework.

laborie2008a; International audience; The multiplication of presentation contexts (such as mobile phones, PDAs) for multimedia documents requires the adaptation of document specifications. In an earlier work, a semantic approach for multimedia document adaptation was proposed. This framework deals with the semantics of the document composition by transforming the relations between multimedia objects. In this chapter, we apply the defined framework to the hypermedia dimension of documents, i.e., hypermedia links between multimedia objects. By considering hypermedia links as particular objects of the document, we adapt the hypermedia dimension with the temporal dimension. However, due to the non-deterministic character of the hypermedia structure, the document is organized in several loosely dependent sub-specifications. To preserve the adaptation framework, we propose a first straightforward strategy that consists of adapting all sub-specifications generated by the hypermedia structure. Nevertheless, this strategy has several drawbacks, e.g., the profile is not able to change between user interactions. Hence, we propose an incremental approach which adapts document sub-specifications step by step according to these interactions. To validate this framework, we adapt real standard multimedia documents such as SMIL documents.

International audience; The World Wide Web can be accessed through a number of different devices, each having its own capabilities and limitations. Additionally, the content of the Web is increasing tremendously in size and variety. Yet, many devices do not embed support for all types of media and formats. Therefore, in order to provide as much information as possible to all kind of devices, media items have to be adapted. In this paper, we propose to adapt them by replacing incompatible media items by others found on the Web. The adapted media items must convey the same message as the original ones, while satisfying the target profile. We present a possible architecture to implement this and we show that search engines can already achieve this to a limited extent. Nonetheless, some results are unsatisfactory because media annotations lack semantics, are partial and are heterogeneous. Hence, we propose to use Semantic Web technologies, such as RDF descriptions, ontologies, ontology merging and matching, in order to select better alternatives, thus improving this adaptation framework.

laborie2006d; International audience; The multiplication of execution contexts for multimedia documents requires the adaptation of document specifications. We proposed a semantic approach for multimedia document adaptation. This paper extends this framework to the hypermedia dimension of multimedia documents, i.e., hypermedia links between multimedia objects. By considering hypermedia links as particular objects of the document, it is possible to adapt the hypermedia dimension with other dimensions like the temporal one. However, due to the hypermedia structure, several specifications have to be considered. Thus, to preserve our adaptation framework, we propose a first straightforward strategy that consists of adapting all specifications generated by the hypermedia structure. However, we show that this one has several drawbacks, e.g., its high computational costs. Hence, we propose to adapt document specifications step by step according to the user interactions.