Search

Present day information retrieval systems largely ignore the issues of lexical and compositional semantics, and rely mainly on some statistical measures for choosing or evolving an indexing scheme. This has been the reason for the decreasing precision in their responses, given an exponentially increasing number of Web pages. The work reported in this paper addresses this issue from a linguistic point of view. We show that the detection of domain-specific phrases can capture the task-specific semantics of documents. We introduce the notion of n*-gram formalism to characterize the domain-specific phrases and their variants, taking a few sample domains. A method to construct a phrase grammar from a small set of documents is proposed. A method of conceptual indexing based on the phrase grammar has also been proposed. In order to demonstrate the effectiveness of the proposed method, we have designed a versatile system that can perform concept-based retrieval, in addition to several document-processing tasks, such as text classification, extraction-based summarization, context tracking, and semantic tagging. Collectively, the system can address the semantic content of documents. Considering the fact that an average user prefers highly relevant results in the top-ranked subset to an exhaustively retrieved set, it is shown that the proposed system performs better in that it retrieves documents that are more conceptually relevant than those retrieved by Google, and at 95% confidence level. Copyright ? 2005 Taylor & Francis Inc.

A story may be analyzed to identify the main characters and to extract the relationship between them. Relation extraction problems are generally solved either through supervised or unsupervised learning algorithms. In the former, there should be a text corpus for which the entities and their relation types are already known. Such algorithms typically learn to classify new entity pairs into any of the relation types it has already seen, based on some recurring patterns. On the other hand, the unsupervised learning approach is used when there is no such marked up corpus. Such algorithms typically identify patterns relevant to the relation extraction task, occurring within the corpus and then use these patterns to group entities such that the entities within a group share similar relationships. The proposed method is a hybrid approach which combines the features of unsupervised and supervised learning methods. It also uses some rules to extract relationships. The method identifies the main characters and collects the sentences related to them. Then these sentences are analyzed and classified to extract relationships. The main applications are story summarization and analysis of the major characters in stories.