Search

Digital preservation encompasses the theory and practice ensuring purposeful future use of digital resources. But how can one tell whether it has been effective or not? The evaluation of preservation efficacy has two dimensions: trustworthiness of managerial programs and systems; and successful use of managed resources. While the former has received extensive attention, the latter has been little investigated. This stems from an insufficiently broad conceptualization of the preservation enterprise, which should be viewed expansively as facilitating meaningful human communication across time and concomitant cultural distance. Communicological analysis leads to a semiotic-phenomenological model for preservation-enabled communication cognizant of the elusive nature of use, which is inherently contingent with respect to time, place, person, and purpose. Preservation success is positioned as an individual, rather than universal value, with a benchmark evaluation of situational verisimilitude, rather than absolute fidelity to an illusory canonical state and information experience. The proposed evaluative approach provides new conceptual clarity to preservation theory and practice, a more rigorous basis for illuminating the limits of preservation efficacy, and a more nuanced means of stating, measuring, and evaluating preservation intentions, expectations, and outcomes.

Digital information is indispensable to contemporary commerce, culture, science, and education. No future understanding of a prior time in the digital age is possible without proactive preservation of our digital heritage. But how can one know whether or not that preservation has been effective? There are two primary assessments of digital preservation efficacy: trustworthiness of managerial systems and programs, and successful use of preserved resources. The first has received extensive treatment in the literature, but the second has been little investigated. This stems from a too narrow conceptualization of the preservation domain as synonymous with data management. Given that the goal of that management is to facilitate future use, and that use is inherently contingent with respect to time, place, person, and purpose, digital preservation should be seen more broadly as facilitating human communication across time. My research asks what measures can meaningful evaluate the efficacy of such communicative acts. It proposes a communicological theory in which success is evaluated with regard to situational verisimilitude. Evaluation metrics are derived from a semiotic-phenomenological model of preservation-enabled communication and the affordances supported by preserved digital resources. This work contributes new conceptual clarity to the theory and practice of digital preservation, a more rigorous basis for demarcating the limits of preservation efficacy, and a more nuanced means of stating, measuring, and evaluating intentions, expectations, and outcomes.

A presentation to staff of the California Digital Library, providing an update on Cobweb development progress as of January, 2018.

The massive scale of online news calls for collaborative approaches to assessing what has already been archived and asserting which un-archived sites should be preserved, considering their significance in representing a “first draft of history.” And given the fast pace at which online news content evolves, memory institutions interested in preserving this content may need to understand not just what has been archived, but also the temporal and structural conditions that were applied in its archiving. Researchers and journalists likewise have an interest in understanding what online news content is available to them, and are well-positioned to make recommendations about which content should be preserved, why, and at what pace, so that it can be made available into the future. There is therefore a potential for highly productive synergies between the news archiving community and the Cobweb initiative, which is building a collaborative collection development platform supporting the creation of comprehensive web archives by coordinating the independent activities of the web archiving community.   The demands of archiving the web in comprehensive breadth or thematic depth easily exceed the capacity of any single institution, and the same could be said of online news content. To ensure that the limited resources of a given archival program are deployed most effectively, it is important that its curators know something about the collection development priorities and holdings of other, similarly engaged institutions. Cobweb, https://github.com/CobwebOrg/cobweb, will meet this need by supporting three key functions of collaborative collection development of web archives: nominating, claiming, and holdings. The nomination function will let curators and stakeholders (including researchers and journalists) suggest web sites pertinent to specific thematic areas and provide seed-level descriptive metadata; the claiming function will allow archival programs to indicate an intention to capture some subset of nominated sites; and the holdings function will allow programs to document captured sites along with their collection-level description, structural and temporal scope, preservation policies, and terms of use. The aggregated  descriptions of websites archived by many distributed archival programs that will be made discoverable through Cobweb are also intended to make it easier for researchers and journalists to find versions of websites relevant to their work.   Cobweb is a collaborative project of the California Digital Library, Harvard University, and UCLA, funded by the Institute for Museum and Library Services. This presentation provides an update on recent project activities, which include finalizing data models and functional requirements, user experience/interface design, development of a prototype interface and database, and exploring interactions of the Cobweb system with other web archiving systems.

The demands of archiving the web in comprehensive breadth or thematic depth easily exceed the capacity of any single institution.  To ensure that the limited resources of a given archival program are deployed most effectively, it is important that its curators know something about the collection development priorities and holdings of other, similarly-engaged institutions.  Cobweb, https://github.com/CobwebOrg/cobweb, will meet this need by supporting three key functions: nominating, claiming, and holdings.  The nomination function will let curators and stakeholders suggest web sites pertinent to specific thematic areas and provide seed-level descriptive metadata; the claiming function will allow archival programs to indicate an intention to capture some subset of nominated sites; and the holdings function will allow programs to document captured sites along with their collection-level description, structural and temporal scope, preservation policies, and terms of use.  Cobweb is a collaborative project of the CDL, Harvard University, and UCLA.  This presentation provides an update on project activities, as of July 2017.

The DMPTool and DMPonline were developed to meet an emerging need arising from the advent of open data policies and each is now well established as the resource for researchers seeking guidance in creating data management plans (DMPs) in the US and UK respectively. Both services, and their sponsoring organizations, the California Digital Library (CDL) and the Digital Curation Centre (DCC), have succeeded in enabling researchers to comply with funder requirements in producing DMPs. However, this is just one step along the road to advancing open science.We see an opportunity to further leverage DMPs to support open science by integrating them into the broader ecosystem of data management infrastructure. In order to achieve this goal, we must redefine success to include not just adoption of our services by institutions but also widespread adoption by individual researchers, disciplinary communities, and funders. Working together with all stakeholders to make DMPs an essential, open part of the research lifecycle, and not just a matter of compliance, is the next step toward effectively managing and sharing research data.We propose to join forces and build a new, global data management advisory platform that links DMPs to other components of the research lifecycle. The biomedical research community provides an opportunity to adapt the infrastructure and associated educational resources to one specific disciplinary community and plug into new initiatives. We will reposition DMPs as living documents useful for structuring the course of biomedical research activities and integrating with related data management systems to lower the barriers for implementation and promote culture change. Consolidating around a single platform for DMPs extends our reach, keeps costs down, and moves best practices forward, allowing us to participate in a truly global open science ecosystem.

Scholars at the ten campuses of the University of California system, like their academic peers elsewhere, increasingly are being asked to ensure that data resulting from their research and teaching activities are subject to effective long-term management, public discovery, and retrieval. The new academic imperative for research data management (RDM) stems from mandates from public and private funding agencies, pre-publication requirements, institutional policies, and evolving norms of scholarly discourse. In order to meet these new obligations, scholars need access to appropriate disciplinary and institutional tools, services, and guidance. When providing help in these areas, it is important that service providers recognize the disparity in scholarly familiarity with data curation concepts and practices. While the UC Curation Center (UC3) at the California Digital Library supports a growing roster of innovative curation services for University use, most were intended originally to meet the needs of institutional information professionals, such as librarians, archivists, and curators. In order to address the new curation concerns of individual scholars, UC3 realized that it needed to deploy new systems and services optimized for stakeholders with widely divergent experiences, expertise, and expectations. This led to the development of Dash, an online data publication service making campus data sharing easy. While Dash gives the appearance of being a full-fledged repository, in actuality it is only a lightweight overlay layer that sits on top of standards-compliant repositories, such as UC3’s existing Merritt curation repository. The Dash service offers intuitive, easy-to-use interfaces for dataset submission, description, publication, and discovery. By imposing minimal prescriptive eligibility and submission requirements; automating and hiding the mechanical details of DOI assignment, data packaging, and repository deposit; and featuring a streamlined, self-service user experience that can be integrated easily into scholarly workflows, Dash is an important new service offering with which UC scholars can meet their RDM obligations.

Scientific datasets have immeasurable value, but they lose their value overtime without proper documentation, long-term storage, and easy discovery andaccess. Across disciplines as diverse as astronomy, demography, archeology,and ecology, large numbers of small heterogeneous datasets (i.e., the long tailof data) are especially at risk unless they are properly documented, saved, andshared. One unifying factor for many of these at-risk datasets is that they residein spreadsheets.In response to this need, the California Digital Library (CDL) partneredwith Microsoft Research Connections and the Gordon and Betty MooreFoundation to create the DataUp data management tool for Microsoft Excel.Many researchers creating these small, heterogeneous datasets use Excel atsome point in their data collection and analysis workflow, so we were interestedin developing a data management tool that fits easily into those work flows andminimizes the learning curve for researchers.The DataUp project began in August 2011. We first formally assessedthe needs of researchers by conducting surveys and interviews of our targetresearch groups: earth, environmental, and ecological scientists. We foundthat, on average, researchers had very poor data management practices, werenot aware of data centers or metadata standards, and did not understand thebenefits of data management or sharing. Based on our survey results, wecomposed a list of desirable components and requirements and solicitedfeedback from the community to prioritize potential features of the DataUp tool.These requirements were then relayed to the software developers, and DataUpwas successfully launched in October 2012.

Scholarly researchers today are increasingly required to engage in a range of data management planning activities to comply with institutional policies, or as a precondition for publication or grant funding. The latter is especially true in the U.S. in light of the recent White House Office of Science and Technology Policy (OSTP) mandate aimed at maximizing the availability of all outputs – data as well as the publications that summarize them – resulting from federally-funded research projects.To aid researchers in creating effective data management plans (DMPs), a group of organizations – California Digital Library, DataONE, Digital Curation Centre, Smithsonian Institution, University of Illinois Urbana-Champaign, and University of Virginia Library – collaborated on the development of the DMPTool, an online application that helps researchers create data management plans. The DMPTool provides detailed guidance, links to general and institutional resources, and walks a researcher through the process of generating a comprehensive plan tailored to specific DMP requirements. The uptake of the DMPTool has been positive: to date, it has been used by over 6,000 researchers from 800 institutions, making use of more than 20 requirements templates customized for funding bodies.With support from the Alfred P. Sloan Foundation, project partners are now engaged in enhancing the features of the DMPTool. The second version of the tool has enhanced functionality for plan creators and institutional administrators, as well as a redesigned user interface and an open RESTful application programming interface (API).New administrative functions provide the means for institutions to better support local research activities. New capabilities include support for plan co-ownership; workflow provisions for internal plan review; simplified maintenance and addition of DMP requirements templates; extensive capabilities for the customization of guidance and resources by local institutional administrators; options for plan visibility; and UI refinements based on user feedback and focus group testing. The technical work undertaken for the DMPTool Version 2 has been accompanied by a new governance structure and the growth of a community of engaged stakeholders who will form the basis for a sustainable path forward for the DMPTool as it continues to play an important role in research data management activities.

Detecting cracks in teeth is a long-standing clinical challenge. Patients may complain of diffuse pain on chewing, pain, at times, on temperature change and pain that occurs episodically. Common diagnostic tools such as radiographs and visual examination may not detect cracks. This clinical case study shows how photothermal radiometry and luminescence (PTR-LUM), technology behind the Canary Dental Caries Detection System can detect and monitor cracks clinically as well as quantify the extent of crack. This important clinical feature is not yet available with other caries detection clinical devices. In this clinical situation, the cracks involved a large part of the mesial and distal of a mandibular second molar and the adjacent first molar. It led to a diagnosis of parafunction and placement of a mandibular flat plane bite splint along with the placement of composite restorations to restore the fractures. The science behind the point scan lock-in signal processing results of PTR-LUM technology implemented in The Canary System to clinically detect visible cracks or cracks beneath the enamel surface as well as caries on all tooth surfaces and around restorations is discussed. Amplitude and phase results from PTR-LUM point scans are incorporated into a Canary number output developed for oral health providers and are disclosed for the first time in detail with clinical evidence. [ABSTRACT FROM AUTHOR]

There is a need to establish a new publishing paradigm to cope with the deluge of data artifacts produced by data-intensive science, many of which are vital to data re-use and verification of published scientific conclusions. Due to the limitations of traditional publishing, most of these artifacts are not usually disseminated, cited, or preserved. These latent artifacts consist largely of datasets and data processing information that together form the foundations of the reasoned analyses that appear in the published literature. But this traditional record of science increasingly represents only the tip of the scientific iceberg.One promising approach to this problem of data invisibility is to wrap these artifacts in the metaphor of a “data paper”, a somewhat unfamiliar bundle of scholarly output with a familiar facade. As envisioned, a data paper minimally consists of a cover sheet and a set of links to archived artifacts. The cover sheet contains familiar elements such as title, authors, date, abstract, and persistent identifier (e.g., a DOI or ARK) — just enough to permit basic exposure to and discovery of data by internet search engines; also just enough to build a basic data citation, to instill confidence in the identifier’s stability, and to be picked up by indexing services such as Google Scholar.This simple format represents only the first stage of the evolution of the data paper. There is room for the format to increase in complexity with the incorporation of other valuable elements, both general-purpose and discipline-specific, to enrich discovery, re-use, and archiving. An exciting potential outcome of this development of the data paper as publication is the parallel emergence of a new kind of “data journal”. Like regular journals, data journals would spring up around disciplines and sub-disciplines as needed, and we could expect that some of them would also be peer-reviewed. The data journal is envisioned as an “overlay” journal; an editor would assemble an issue by selecting data papers from any number of source and archives, and combining them with front matter, a table of contents, editorial policies, submission guidelines, etc.This new data publishing paradigm promises to strengthen the scientific community practices of data sharing, re-use, and preservation. Scientists want to do science, get credit for it, communicate about it with their peers, and improve the measurable outputs by which their funders and employers evaluate their performance. The elements of the data paper create a recognizable and standardized form for previously unpublished data artifacts, making them easier to approach, evaluate, and automatically index for basic discovery purposes. Those same elements can easily be repurposed to create familiar-looking citations suitable for reference in CVs and all manner of publication. Finally, unique persistent identifiers for data papers and data artifacts greatly facilitate automatic discovery of a data paper’s impact and re-use.

The effective long-term curation of digital content requires expert analysis, policy setting, and decision making, and a robust technical infrastructure that can effect and enforce curation policies and implement appropriate curation activities. Since the number, size, and diversity of content under curation management will undoubtedly continue to grow over time, and the state of curation understanding and best practices relative to that content will undergo a similar constant evolution, one of the overarching design goals of a sustainable curation infrastructure is flexibility. In order to provide the necessary flexibility of deployment and configuration in the face of potentially disruptive changes in technology, institutional mission, and user expectation, a useful design metaphor is provided by the Unix pipeline, in which complex behavior is an emergent property of the coordinated action of a number of simple independent components. The decomposition of repository function into a highly granular and orthogonal set of independent but interoperable micro-services is consistent with the principles of prudent engineering practice. Since each micro-service is small and self-contained, they are individually more robust and collectively easier to implement and maintain. By being freely interoperable in various strategic combinations, any number of micro-services-based repositories can be easily constructed to meet specific administrative or technical needs. Importantly, since these repositories are purposefully built from policy neutral and protocol and platform independent components to provide the function minimally necessary for a specific context, they are not constrained to conform to an infrastructural monoculture of prepackaged repository solutions. The University of California Curation Center has developed an open source micro-services infrastructure that is being used to manage the diverse digital collections of the ten campus University system and a number of non-university content partners. This paper provides a review of the conceptual design and technical implementation of this micro-services environment, a case study of initial deployment, and a look at ongoing micro-services developments.

In order better to meet the needs of its diverse University of California constituencies, the California Digital Library UC Curation Center is re-envisioning its approach to digital curation infrastructure by devolving function into a set of granular, independent, but interoperable micro-services. Since each of these services is small and self-contained, they are more easily developed, deployed, maintained, and enhanced; at the same time, complex curation function can emerge from the strategic combination of atomistic services. The emergent approach emphasizes the persistence of content rather than the systems in which that management occurs, thus the paradigmatic archival culture is not unduly coupled to any particular technological context. This results in a curation environment that is comprehensive in scope, yet flexible with regard to local policies and practices and sustainable despite the inevitability of disruptive change in technology and user expectation.

The JHOVE characterization framework is widely used by international digital library programs and preservation repositories. However, its extensive use over the past four years has revealed a number of limitations imposed by idiosyncrasies of design and implementation. With funding from the Library of Congress under its National Digital Information Infrastructure Preservation Program (NDIIPP), the California Digital Library, Portico, and Stanford University are collaborating on a two year project to develop and deploy a next-generation architecture providing enhanced performance, streamlined APIs, and significant new features. The JHOVE2 project generalizes the concept of format characterization to include identification, validation, feature extraction, and policy-based assessment. The target of this characterization is not a simple digital file, but a (potentially) complex digital object that may be instantiated in multiple files.

The Digital Preservation Program of the California Digital Library (CDL) is engaged in a process of reinvention involving significant transformations of its outlook, effort, and infrastructure. This includes a re-articulation of its mission in terms of digital curation, rather than preservation; encouraging a programmatic, rather than a project-oriented approach to curation activities; and a renewed emphasis on services, rather than systems. This last shift was motivated by a desire to deprecate the centrality of the repository as place. Having the repository as the locus for curation activity has resulted in the deployment of a somewhat cumbersome monolithic system that falls short of desired goals for responsiveness to rapidly changing user needs and operational and administrative sustainability. The Program is pursuing a path towards a new curation environment based on the principle of devolving curation function to a set of small, simple, loosely-coupled services. In considering this new infrastructure the Program is relying upon a highly deliberative process starting from first principles drawn from library and archival science. This is followed by a stagewise progressesion of identifying core preservable values, devising strategies promoting those values, defining abstract services embodying those strategies, and, finally, developing systems that instantiate those services. This paper presents a snapshot of the Program’s transformative efforts in its early phase.