Search

The Authority and Ranking Effects play a key role in data fusion. The former refers to the fact that the potential relevance of a document increases exponentially as the number of systems retrieving it increases and the latter to the phenomena that documents higher up in ranked lists and found by more systems are more likely to be relevant. Data fusion methods commonly use all the documents returned by the different retrieval systems being compared. Yet, as documents further down in the result lists are considered, a document's probability of being relevant decreases significantly and a major source of noise is introduced. This paper presents a systematic examination of the Authority and Ranking Effects as the number of documents in the result lists, called the list depth, is varied. Using TREC 3, 7, 8, 12 and 13 data, it is shown that the Authority and Ranking Effects are present at all list depths. However, if the systems in the same TREC track retrieve a large number of relevant documents, then the Ranking Effect only begins to emerge as more systems have found the same document and/or the list depth increases. It is also shown that the Authority and Ranking Effects are not an artifact of how the TREC test collections have been constructed. Keywords: Data fusion; Meta search; Authority; Ranking effects

Document classification, with the blooming of the Internet information delivery, has become indispensable required and is expected to be disposed by an automatic text categorization. This paper presents a text categorization system to solve the multi-class categorization problem. The system consists of two modules: the processing module and the classifying module. In the first module, ICF and Uni are used as the indictors to extract the relevant terms. While the fuzzy set theory is incorporated into the OAA-SVM in the classifying module, we specifically propose an OAA-FSVM classifier to implement a multi-class classification system. The performances of OAA-SVM and OAA-FSVM are evaluated by macro-average performance index. Also the statistical significance test is examined by the McNemar's test. The results from the empirical study show that the proposed OAA-FSVM method has out-performed OAA-SVM in the multi-class text categorization problem. Keywords: Fuzzy support vector machine; Feature selection: Membership functions; OAA-FSVM

This paper presents a cluster validation based document clustering algorithm, which is capable of identifying an important feature subset and the intrinsic value of model order (cluster number). The important feature subset is selected by optimizing a cluster validity criterion subject to some constraint. For achieving model order identification capability, this feature selection procedure is conducted for each possible value of cluster number. The feature subset and the cluster number which maximize the cluster validity criterion are chosen as our answer. We have evaluated our algorithm using several datasets from the 20Newsgroup corpus. Experimental results show that our algorithm can find the important feature subset, estimate the cluster number and achieve higher micro-averaged precision than previous document clustering algorithms which require the value of cluster number to be provided. Keywords: Document clustering; Cluster validation; Feature selection; Cluster number estimation

Search engines fail to make a clear distinction between items of varying relevance when presenting search results to users. Instead, they rely on the user of the system to estimate which items are relevant, partially relevant, or not relevant. The user of the system is given the task of distinguishing between documents that are relevant to different degrees. This process often hinders the accessibility of relevant or partially relevant documents, particularly when the results set is large and documents of varying relevance are scattered throughout the set. In this paper, we present a clustering scheme that groups documents within relevant, partially relevant, and not relevant regions for a given search. A clustering algorithm accomplishes the task of clustering documents based on relevance. The clusters were evaluated by end-users issuing categorical, interval, and descriptive relevance judgments for the documents returned from a search. The degree of overlap between users and the system for each of the clustered regions was measured to determine the overall effectiveness of the algorithm. This research showed that clustering documents on the Web by regions of relevance is highly necessary and quite feasible.

Imagine this too-frequent problem: A company has paper coming in at different times, and too much time is spent matching received documents to invoices and ultimately to purchase orders. This […]

Document Image Processing (DIP) and Optical Character Recognition (OCR) are being used by the UK Employment Department (ED) to speed up the processing of the 1993 Census of Employment. The Information Systems Branch of the ED implemented the system. Trials were commissioned in Italy and the US prior to procuring the system and Andersen Consulting provided early training. A similar system has been set up to process the UK New Earnings Survey for 1994.

Remember the lecture about simple machines that you heard in your junior high school science class? Inclined plane, wheels, levers, all that good stuff? And, of course, the inevitable story [...]

Office design trends are dictated by technology and the need to increase productivity. In managing document processes, workers divide their time between generating documents through computers and analyzing such documents in hard copy. Workers often have to move from device to device to fulfill these two functions. One potential solution is to develop a system in which both functions can be performed at the same time. The key challenges facing the development of such a system will be managing job contention, ensuring reliability and compatibility with the other devices in the office., The main force driving office design is that the way we work has changed. For example, vacancy rates are often looked upon as an economic bellwether. But high LI vacancy [...]