Search

With an increasing number of people working with large music archives, advanced methods for automatic labeling and organization of music collections are required as these archives grow in size. Manual annotation and categorization is not feasible for massive collections of music. In the research domain of music information retrieval (MIR) a number of algorithms for the content-based description of music were developed, which perform the extraction of relevant features for the computation of similarity between pieces of music. This fundamental step enables a great range of applications for music retrieval and organization. With supervised machine learning, music can be classified into different kinds of categories, such as genres, artists or moods. Using unsupervised approaches such as the self-organizing map music can be clustered by similar style and visualized in a way that enables direct retrieval of similar music at a glance. In this chapter, we will review the most common audio feature extraction techniques, which serve as a basis for subsequent classification and clustering tasks. As an example, we will show how music is classified into a set of genres and how genre classification can be used for benchmarking. Moreover, the creation of the so-called "music maps" and their various visualizations is demonstrated, and an interactive application called "PlaySOM" is presented, with an interface which allows direct access to similar sounding pieces in a large music collection. Its mobile counterpart "PocketSOMPlayer" allows direct playback from a music map on a mobile device without having to browse lists. Both allow the convenient interactive creation of situation-based playlists. [ABSTRACT FROM AUTHOR]

We propose a generative statistical model for the classification of semi-structured multimedia documents. Its main originality is its ability to simultaneously take into account the structural and the content information present in a semi-structured document and also to cope with different types of content (text, image, etc.). We then present the results obtained on two sets of experiments: • One set concerns the filtering of pornographic Web pages • The second one concerns the thematic classification of Wikipedia documents. [ABSTRACT FROM AUTHOR]

Semantic labeling of large volumes of image and video archives is difficult, if not impossible, with the traditional methods due to the huge amount of human effort required for manual labeling used in a supervised setting. Recently, semi-supervised techniques which make use of annotated image and video collections are proposed as an alternative to reduce the human effort. In this direction, different techniques, which are mostly adapted from information retrieval literature, are applied to learn the unknown one-to-one associations between visual structures and semantic descriptions. When the links are learned, the range of application areas is wide including better retrieval and automatic annotation of images and videos, labeling of image regions as a way of large-scale object recognition and association of names with faces as a way of large-scale face recognition. In this chapter, after reviewing and discussing a variety of related studies, we present two methods in detail, namely, the so called "translation approach" which translates the visual structures to semantic descriptors using the idea of statistical machine translation techniques, and another approach which finds the densest component of a graph corresponding to the largest group of similar visual structures associated with a semantic description. [ABSTRACT FROM AUTHOR]

In comparison with the classic query-by-example paradigm, the "mental image search" paradigm lifts the strong assumption that the user has a relevant example at hand to start the search. In this chapter, we review different methods that implement this paradigm, originating from both the content-based image retrieval and the object recognition fields. In particular, we present two complementary methods. The first one allows the user to reach the target mental image by relevance feedback, using a Bayesian inference. The second one lets the user specify the mental image visual composition from an automatically generated visual thesaurus of segmented regions. In this scenario, the user formulates the query with an explicit representation of the image content, as opposed to the first scenario which accommodates an implicit representation. In terms of usage, we will show that the second approach is particularly suitable when the mental image has a well-defined visual composition. On the other hand, the Bayesian approach can handle more "semantic" queries, such as emotions for which the visual characterization is more implicit. [ABSTRACT FROM AUTHOR]

In recent years there has been a growing interest in improving all aspects of the interaction between humans and computers with the clear goal of achieving a natural interaction, similar to the way human-human interaction takes place. The most expressive way humans display emotions is through facial expressions. Humans detect and interpret faces and facial expressions in a scene with little or no effort. Still, development of an automated system that accomplishes this task is rather difficult. There are several related problems: detection of an image segment as a face, extraction of the facial expression information, and classification of the expression (e.g., in emotion categories). A system that performs these operations accurately and in real time would be a major step forward in achieving a human-like interaction between the man and machine. In this chapter, we present several machine learning algorithms applied to face analysis and stress the importance of learning the structure of Bayesian network classifiers when they are applied to face and facial expression analysis. [ABSTRACT FROM AUTHOR]

In this chapter we will introduce a new effective framework for learning an object detector. The main idea is to minimize the manual effort when learning a classifier and to combine the power of a discriminative classifier with the robustness of a generative model. Starting with motion detection an initial set of positive examples is obtained by analyzing the geometry (aspect ratio) of the motion blobs. Using these samples a discriminative classifier is trained using an online version of AdaBoost. In fact, applying this classifier nearly all objects are detected but there is a great number of false positives. Thus, we apply a generative classifier to verify the obtained detections and to decide if a detected patch represents the object of interest or not. As we have a huge amount of data (video stream) we can be very conservative and use only patches for (positive or negative) updates if we are very confident about our decision. Applying this update rules, an incrementally better classifier is obtained without any user interaction. Moreover, an already trained classifier can be retrained online and can therefore easily be adapted to a completely different scene. We demonstrate the framework on different scenarios including pedestrian and car detection. [ABSTRACT FROM AUTHOR]

Content-based image retrieval (CBIR) has attracted a lot of interest in recent years. When considering visual information retrieval in image databases, many difficulties arise. Learning is definitively considered as a very interesting issue to boost the efficiency of information retrieval systems. Different strategies, such as offline supervised learning or semi-supervised learning, have been proposed. Active learning methods have been considered with an increased interest in the statistical learning community. Initially developed in a classification framework, a lot of extensions are now proposed to handle multimedia applications. The purpose of this chapter is to present an overview of the online image retrieval systems based on supervised classification techniques. This chapter also provides algorithms in a statistical framework to extend active learning strategies for online content-based image retrieval. [ABSTRACT FROM AUTHOR]

When data objects that are the subject of analysis using machine learning techniques are described by a large number of features (i.e. the data are high dimension) it is often beneficial to reduce the dimension of the data. Dimension reduction can be beneficial not only for reasons of computational efficiency but also because it can improve the accuracy of the analysis. The set of techniques that can be employed for dimension reduction can be partitioned in two important ways; they can be separated into techniques that apply to supervised or unsupervised learning and into techniques that either entail feature selection or feature extraction. In this chapter an overview of dimension reduction techniques based on this organization is presented and the important techniques in each category are described. [ABSTRACT FROM AUTHOR]

Unsupervised learning is very important in the processing of multimedia content as clustering or partitioning of data in the absence of class labels is often a requirement. This chapter begins with a review of the classic clustering techniques of k-means clustering and hierarchical clustering. Modern advances in clustering are covered with an analysis of kernel-based clustering and spectral clustering. One of the most popular unsupervised learning techniques for processing multimedia content is the self-organizing map, so a review of self-organizing maps and variants is presented in this chapter. The absence of class labels in unsupervised learning makes the question of evaluation and cluster quality assessment more complicated than in supervised learning. So this chapter also includes a comprehensive analysis of cluster validity assessment techniques. [ABSTRACT FROM AUTHOR]