NDSR: Designing Digital Archive Systems at The National Security Archive
Who are digital stewards of the future? How do we best prepare them for a career collecting, managing, preserving, and making accessible our digital assets? How will they gain the required layers of expertise in legacy systems, business management, programming, software, data analysis, project management, and strategic planning required to do so? The problems inherent in managing mutable and ephemeral digital records are not going away any time soon…who will provide the solutions in five, ten, or twenty years?
The National Digital Stewardship Residency (NDSR)
These are the questions that the National Digital Stewardship Residency was created to answer. The NDSR program is an initiative of the IMLS and the Library of Congress to bring ten young professionals identified as leaders in their field into the professional community of digital librarians and archivists in the DC area, where they can best develop and apply their knowledge of digital stewardship. Each resident is assigned to a local institution and given a specific project that matches and advances their skills in a particular area. Project sites include the Library of Congress, the Folger Shakespeare Library, the World Bank, and Dumbarton Oaks, to name a few, and the projects range from planning for digitization of analog moving image collections to developing institutional policies for appraisal of digital content, each project unique and uniquely challenging. You can see a full list of the projects here, a list of our Twitter activity here, and list of the resident’s individual blogs here.
Project Site: The National Security Archive
My project site is The National Security Archive, a small non-profit institution dedicated to a mission of usable history, human rights, and a more open government, and holding hundreds of thousands of digital government documents collected through FOIA requests and analyst research. The Archive, like many of its peer institutions, was founded at a time when all of the documents it worked with were analog, and now finds itself working with documents that are almost entirely born-digital or previously digitized. The transition has not always been smooth and the Archive has ended up with five legacy systems governing its digital records, an unclear knowledge what it has and where it is, and ad-hoc procedures and workflows for digital asset management. I was brought in by the Archive and the NDSR program to help the Archive’s staff analyze and inventory their records, analyze and better understand their digital asset management systems, procedures, and institutional culture, and based on that understanding, propose a strategic plan for improved digital content management in the future.
Digital Asset Assessment
The first phase of the project entailed analyzing the digital assets and digital asset management systems at the Archive. I took the first month and half to explore these two areas, diving into individual collections and documents as well as the groups and tracking documents that did exist, though those were few and far between. The Archive’s assets were split, as I said before, between five legacy systems. These systems included two I had never worked with, CuadraSTAR and Alchemy, as well as two I was more familiar with, a static HTML website being used for content management and a Microsoft Access database. The fifth system being used for digital asset management was a combination of the Archive’s shared and local drives, an informal arrangement that nonetheless held a vast quantity of data, some of it the only copy of a particular document set.
As part of my exploration and analysis I created a simple index, in Excel, of all of the Archive’s collections. I then collected further details about certain document sets, particularly those on the website and those in Alchemy, since these tended to be the most valuable (that is, complete, curated, and often-used) collections.
Understanding Stakeholders and Institutional Needs
The second phase of the project was to identify and get to know the primary stakeholders and existing procedures in the digital asset management process. My conversations with the stakeholders helped me to understand their current uses of digital records, their functional needs for any digital record management systems, and their potential needs – how they could imagine an ideal digital asset management system helping them in their work. I cannot overemphasize the importance of this phase, since each facet of any proposed improvement plan must emerge from and closely attend to the needs of the project’s stakeholders in order to stand any chance at securing buy-in and future use or maintenance.
At the National Security Archive, the primary stakeholders included the director and co-director, the current head of IT, the webmaster, and the analysts that actively worked with digital documents and records, including the CuadraSTAR, Access, and Alchemy teams as well as several individual project analysts. I interviewed, observed, re-interviewed, and sent clarifying questions to each of these people multiple times over, building an understanding of how the Archive currently used and how it would eventually like to use its digital assets from the ground up, so that I could better design plans that suited the Archive’s needs and working styles.
Drafting the Initial Plan
The third phase of the project was also the least quantifiable. It was time to put pen to paper and begin drafting the existing workflows, procedures, and systems, and then to begin drafting a “new” system – outlining places for improvement, replacement, reorganization, policy development, workflow development, and risk mitigation, and then outlining how each of those items might unfold as an individual project. This period of drafting overlapped with some activities of the earlier phases, and called for reiteration of some of the information gathering activities, as well as additional research into ideas and conclusions that had emerged from the digital asset, system, and stakeholder analysis.
What finally emerged was what I called an “Assessment Report,” which I’ve outlined in a bit more detail here. To summarize, I didn’t want to just begin writing plans…I wanted any plan, any project, to fit in with the broader Archive’s culture and functional needs. Therefore, the report began by outlining how I saw the Archive’s mission, how its digital assets helped it fulfill that mission, how they could perhaps better do so by making a few small changes, and then how each of those changes might look as bite-sized, individual projects. The report ended with four different plan proposals, ranging from the most simple (improving current systems) to the most complex (designing a new system).
The assessment report was sent to the Archive’s directors at the end of December. The next step will be discussing and revising the report and proposed plans with them, ideally concluding with the creation of a single plan, after which I can build the proposal into a more formal project management plan, break it into manageable stages, and begin taking the first steps, which would most likely be the creation of institution-wide digital asset management policies.
Presenting Our Work
The National Digital Stewardship Residents will be presenting our work at the ALA Midwinter Meeting in Philadelphia, at the ALA Midwinter Meeting in Philadelphia and the IMLS WebWise conference in Baltimore. Three of us, myself included, will likely attend and possibly present at the CNI Spring 2014 Membership Meeting in St. Louis, and another few of us will be at the Code4Lib conference in Raleigh in March. If you’re interested in meeting any of us or learning more about our projects, please feel free to find us at one of those conferences, at Twitter, or on our individual websites.