April 21, 2006

Executive Summary

A group of technical people in the Cornell University Library (CUL) system evaluated several solutions for managing persistent identifiers for the digital resources in three of CUL's digital collection projects. After having compared those solutions' capabilities with the group's list of requirements, the members of the group created a list of recommendations for a CUL strategy, including the adoption of the CNRI Handle System for a test implementation.

The group and its charge

 Report of the Working Group to Evaluate Persistent Identifiers for Digital Objects at CUL

 April 20, 2006

Charge

Bill Kehoe and Rick Silterra conceived of the need for this group while discussing their desire for a consistent strategy for global and persistent identifiers (PIDs) for digital objects. They recognized the integral role that PIDs would play in several ongoing project, including Priority Teams One and Two and the MathArc projectthe Integrated Framework, CUL OAIS, and MathArc projects. They believed the best way to achieve consensus for a strategy would be through a lightweight, focused, and efficient evaluation process. They hoped to assemble a small group of people designing  systems that use identifiers, who could vet the different approaches to PIDs with the goal of finding the most appropriate PID strategy for use on these projects. Marcy Rosenkrantz, Nancy McGovern and Oya Reiger proceeded to help organize a group of people to carry out the evalutation process.  It included (in alphabetical order):

...

• Digital collection identifiers
  • A long standing issue in the presentation of our digital collections is the lack of a reliable identifier
    for the collections themselves. Collections move to new machines, collections change their delivery platform,
    and collections change their default behaviors. We need to be able to locate collections reliably over a long period of time and discover aspects of collections for interoperating with other collections.

• Persistent IDs for digital preservation  preservation
  
  • We are working on preserving the digital objects in two large collections, the Euclid journals and the arXiv.org preprints.  In general, we can view the objects as having at least one content file and an object descriptor file containing metadata about the object. Most digital objects contain multiple content and metadata files.  We need to be able to identify and locate the files for a long time, regardless of where they are located. Rather than changing the metadata in the descriptor file every time a file is moved-we would like to create persistent identifiers that can be mapped to the files' current locations.
    
  • The number of component files to be preserved will be several times larger than  the number of digital objects. With processing efficiency in mind, we would prefer a solution that will allow us to resolve the identifiers locally, without going out over the internet for each request for resolution.
    
  • The digital objects' component files in our preservation system will not be directly accessible to the public; access will occur through a gated interface. The persistent identifier mechanism we use should be able to be produce identifiers that are private and not discoverable.
    

Requirements for an implementation

• We don't want to break any system currently used at CUL that uses persistent identifieres, such as the PURL server. It should be backward compatible.
• Every PID should be globally unique.
  
• PIDs must be globally resolvable, resolvable from anywhere.
  

• On the other hand, some PIDs must be optionally resolvable only within a constrained environment. In other words, the system must have the ability to make PIDs private, not discoverable.  For example, an archiving system should be able to restrict access to  low-level components of complex digital objects.

• The system should be able to ensure confidentiality. It should define a mechanism for client authentication and authorization to ensure data integrity and authority control.
• The system shoud should not have dependencies on external systems in order to resolve local PIDs. Reliance on the network outside CUL for every resolution request would introduce potential latency problems, slowing down all our systems.
• PID should be free of location semantics. For example, a file originally stored on a server as "/some/graphic/collection/file1234" should not be mapped to a PID like "cul:graphics_collection_00000001234."  If ownership of the file changes, or location changes, or if the file is reassigned to another collection, the PID would have misleading and incorrect meaning.  Any relationships among files, collections, and owners should be documented in metadata.

...

• an overview of identifier strategies and issues,
• PURL,
• ARK,Handle,
• Handl
• OpenURL, and
• OAI-PMH requirements.

By the end of these discussions, the group began to reach a consensus around Handles based on that approach's general maturity and installed base, its fit for CUL's projects, and the existing knowledge about this approach within CUL. The group then began to plan and write this report on the group Wiki that recommends further exploration of Handles.

   
 Recommendations

• We recommend that the Library use the CNRI Handles System for generating and maintaining persistent identifiers at Cornell.
  • The Corporation for National Research Initiatives (CNRI) undertakes, fosters, and promotes research in the public interest. One of its many projects is the Handle System.
  • The Handle System is a comprehensive system for assigning, managing, and resolving persistent identifiers, known as "handles," for digital objects and other resources on the Internet. Handles can be used as Uniform Resource Names (URNs).
  • The Handle System includes an open set of protocols, a namespace, and an implementation of the protocols. The protocols enable a distributed computer system to store handles of digital resources and resolve those handles into the information necessary to locate and access the resources. This associated information can be changed as needed to reflect the current state of the identified resource without changing the handle, allowing the name of the item to persist over changes of location and other state information. Each handle may have it own administrator(s), and administration can be done in a distributed environment. The name-to-value bindings may also be secured, allowing handles to be used in trust management applications.
  • The system operates by assigning a unique identifier, or handle, to every file sought to be retrieved by the system. The handle and the file's location are then registered with a handle server. The handle server can either be private, on a local network, or public, as on the Internet. When a user requests a document by submitting its handle, the handle server reports the location of the file, which is then retrieved and presented to the user.
  • The Handles System naming convention is sufficiently flexible to allow truly unique names for digital objects. Handles take the form: naming authority/name. Naming authorities are organized in a hierarchy, with each being able to create sub-naming authorities. Document names need only be unique with reference to the issuing naming authority.
• We recommend that CUL undertake a proof-of-concept implementation of the CNRI Handle System. We would hope to test the system's ability to meet all of our requirements and gain insight and skill in the technical and organizational demands of maintaining a persistent identifier system.  CNRI provides free handle server software and documentation. We envision a local system that can be administered in a distributed manner, so caretakers of different collections will be able to make autonomous decisions about the identifiers their collections use. We also recognize that a proof-of-concept system might prove to be of limited or no use in the long run.  During the period of experimentation we would ensure that the digital objects we give identifiers to would be able to be remapped to their original URLs or some other useful identifer system, in case we decide to discontinue using the Handle System. 

...

• We recommend that, after a period of testing, some write a Usage Document be written to explain how the system can be integrated into the CUL collection lifecycle. Monitoring how the handles are used will assist CUL to optimize the way in which we make our collections available.

• Finally, we recommend that, embedded in mapping the metadata mapped to each PID, there be an explicit statement of the estimated lifespan of the PID and of the object it represents. This practice will add value to the identifier and help enforce a best practice in the lifecycle management of our CUL's digital objects.