Ingest Strategies for Digital Libraries: the Challenges of Handling Portable Objects.

Increasingly we have come to think of institutional repositories as the primary storage and access locations for the deluge of digital materials. However, we continue to employ portable physical carriers for the storage and transfer of digital information, from portable hard drives, USB and Flash drives, through to DVD-ROMs, CD-ROMs and floppy disks. As concerns about the fragility and obsolescence of physical carriers continues to mount, it is likely that digital material will increasingly be transferred back to digital repository systems where curation of these objects will be simplified. The department of Humanities Advanced Technology and Information Institute (HATII) at the University of Glasgow ran the Packaged Object Ingest Project (POIP) to investigate the requirements, procedures and challenges that arise from ingesting digital objects on portable media into a repository. The Packaged Object Ingest Project provides a first step towards defining best practice for ingest from portable carriers. The work that lies behind this paper was carried out under ERPANET, a project funded by the EU under the FPS programme. The challenges of ingest from portable carriers should not be underestimated. A range of material was selected, designed to provide a realistic testbed. We studied a variety of published and unpublished CD-ROMs, published and unpublished 3.5" floppy disks, and unpublished 5.25" disks. We found the content and variations in production often posed challenging intellectual issues. How should we catalogue and store objects containing complex multimedia software applications or those containing seemingly unrelated collections of documents? Particularly with floppy disks, we encountered examples where the collections spanned multiple disks. What is the most appropriate way to catalogue these objects and retain the collection's provenance? The increased reliance on networked storage and delivery along with the rate of technological change means hardware obsolescence is a tangible risk. We wished to investigate which issues associated with ingest were unique to physical carriers as opposed to network delivered content. Our approach was to develop a small testbed database and repository, focusing on the functions related to ingest of the contents of portable objects. We selected what we then perceived to be the most suitable preservation metadata schema, the National Library of New Zealand Preservation Metadata Schema, as our underlying data structure. Having built a simple interface, we ingested the contents represented on a range portable objects, paying particular attention to address issues related to the variety of content and production values. Unsurprisingly, we discovered that the manual ingest of digital contents is a slow and expensive process, particularly for complex objects represented in many files with varying levels of binding. The array of file formats encountered and range of appraisal decisions meant that any ingest operator would be required to have not only a high level of technical skill, but often familiarity with the subject domain and an appreciation of the relevance of the contents. It was necessary to render the object to appraise it's contents. Attempting to render the digital objects on different hardware and operating systems resulted in either failure, partially broken or severely limited renderings on several occasions. The range of file formats found on most portable objects meant that significant preservation metadata was required and though extraction and creation was expensive at first, the resource cost decreased with repetition. This paper describes the research that we conducted, summarises our findings, and presents the practical guidance that we developed as a result. In conclusion we indicate areas where further research might be appropriate to refine our conclusions and to address questions where our results were less than conclusive. [ABSTRACT FROM AUTHOR]