Search

The report very briefly summarizes the research performed during the contract period March 1, 1964, to February 28, 1971. The emphasis of the research was on the discovery and development of techniques for automatically indexing and classifying documents. The research was limited to statistical techniques rather than semantic or syntactic. A bibliography of technical reports published is included. (Author/SJ)

Experience of browsing with a text retrieval system is reported. The technique discussed consists of adding and deleting query terms on successive interactive cycles. The number of hits consistently increased, while the number of false drops consistently decreased on successive cycles. The search system used allows non-Boolean free form query statements. Therefore, an abstract may be used as a query with the objective of finding similar abstracts from the file. (Author)

A series of experiments was performed to investigate the effectiveness and utility of automatically classifying documents through the use of multiple discriminant functions. Classification is accomplished by computing the distance from the mean vector of each category to the vector of observed frequencies of a document and assigning the document to the category having the highest probability. Data concerning the effect of the principal classification parameters on classification performance is reported, based on a data base of approximately 2700 abstracts from the solid state physics field. The parameters studied were the number of sample documents required to define a category, the length of documents, the inter-relationship of the number of sample documents and their lengths, the relation of the number of word types in a document to the number of categories, and performance measures. A higher performance level was obtained when samples of 140 documents were used to define each category than with samples of 35 and 70 documents. Classification results obtained on independent test sets of documents ranged from 73 to 92 per cent. The test sets contained 419 and 1333 documents. Results are also reported in terms of Swets' effectiveness measure and Cleverdon's ratios of relevance, recall and precision. (Author)

To support studies in automatic indexing, classification and extracting, a general purpose frequency program was developed to further theoretical and practical understanding of text word distributions. While the program is primarily designed for counting strings of character-oriented data, it can be used without change for counting any items which can be represented in an integral number of characters. Counts may be obtained simultaneously at several levels of detail, such as for sentences, paragraphs, chapters and entire documents. Both printed outputs and outputs for further computer processing may be obtained, and a variety of summary and detailed outputs are available. The program, titled FRQNCY, is written in the Fortran IV language and was compiled and run on the IBM System/360 using Fortran IV (G) and the System/360 Operating System. It uses at least one feature of the IBM System/360 Fortran IV language (LOGICAL*1 and INTEGER*2 variables) which is not in USASI Fortran and hence may not compile or run under other Fortran systems. The program is extensively parametrized to allow its efficient use on computers with varying amounts of immediate-access storage and input/output equipment. The report is a complete writeup of the frequency program. It covers the purpose and usage of the program and also describes its organization and internal operation. Guidelines for modifying the program or adapting it to different computers are included. (Author)