Inferring semantics from textual information in multimedia retrieval

Abstract: We propose a method for inferring semantic information from textual data in content-based multimedia retrieval. Training examples of images and videos belonging to a specific semantic class are associated with their low-level visual and aural descriptors augmented with textual features such as frequencies of significant words. A fuzzy mapping of a semantic class in the training set to a class of similar objects in the test set is created by using Self-Organizing Maps (SOMs) trained from the low-level descriptors. Experiments with two databases and different textual features show promising results, indicating the usefulness of the approach in bridging the gap from low-level visual features to semantic concepts. [Copyright &y& Elsevier]