Search

The Web is often characterized as a universal information space, in the sense that the provision of and access to Web content is open to all. It is well known, however, that the majority of Web content originates from a small group of English-speaking countries, chiefly the United States. This paper examines trends in the internationalization of Web content, and in particular, whether the Web is becoming increasingly globalized as it matures, and to what segment of the world's countries and languages it currently extends. Sample data from 1998 and 1999 suggests that the Web's content is still far from a true global cross-section, but is slowly expanding both in terms of its geographical origins and the languages in which it is made available.

Discovery and description of Web resources are currently facilitated by the use of metadata. Although much work has been conducted in developing metadata architectures and standards, few studies have undertaken to assess current metadata usage in Web documents. An analysis of this kind is challenging in view of the difficulty of data collection. A practical approach is to collect a representative sample of Web documents, from which valid inferences can be made concerning the Web as a whole. The Web Characterization Project has developed a methodology for extracting a valid sample from the Web. Using data from a sample collected in June 1998, this paper characterizes the use of metadata in Web documents.

The rapid growth in the number of libraries providing Web access services has created a need for reliable, timely statistics characterizing the content of Web-accessible information. The size of the Web makes it impractical to develop descriptive statistics based on an exhaustive survey. An alternative approach is to collect a representative sample of Web pages. This report describes a methodology for sampling the content of the Web through the use of randomly generated IP addresses.