2020 DLF Forum: Building Community With DLF’s Assessment Interest Group

Join us online! November 9th-13th, 2020.Though DLF is best known for our signature event, the annual DLF Forum, our working groups collaborate year round. Long before COVID-19 introduced the concept of “Zoom fatigue” into our lives, DLF’s working groups organized across institutional and geographical boundaries, building community while apart, to get work done. Made possible through the support of our institutional members, working groups are the efforts of a committed community of practitioners, using DLF as a framework for action, engaged in problem-solving in a variety of digital library subfields from project management and assessment to labor and accessibility.

Once we decided that the 2020 DLF Forum and affiliated events would be held in a virtual format, it meant that our working groups wouldn’t have the opportunity to meet in person for their typical working meals that take place throughout the Forum; however, this year’s virtual format means that we’ll have more new DLF Forum attendees than ever before. Because DLF’s working groups are open to ALL, regardless of whether you’re affiliated with a DLF member institution or not, we asked leaders of the DLF working groups to introduce their groups and the work they do to the new and returning members of the #DLFvillage in a series of blogs and videos.

We’ll share these working group updates in the days leading to this year’s DLF Forum. All of our 2020 events are free of charge. Register by November 1.

Updates from the DLF Assessment Interest Group (AIG)

The DLF Assessment Interest Group (AIG) seeks to engage the community in developing best practices and guidelines for various kinds of digital library assessment. As the amount of digitized/born-digital content continues to grow at our research and cultural heritage institutions, there is an ever-increasing need to strategically standardize our assessment efforts. 

The only requirement for participation in a DLF AIG working group is a willingness to dig in and devote a small part of your time contributing to the tools, methods, and body of knowledge on digital library assessment. We encourage you to join a meeting or reach out to the contacts listed below. (DLF membership is not a requirement for participation.)

We look forward to extending an invitation for you to join us at our upcoming AIG virtual event in early 2021, which will showcase working group achievements and help us brainstorm for the year ahead. Stay tuned for event details! 

In the meantime, here’s an overview of the five AIG working groups and along with some recent highlights. The groups represented in this post are: the Content Reuse Working Group, the Cost Assessment Working Group, the Cultural Assessment Working Group, the Metadata Assessment Working Group, and the User Experience Assessment Working Group.

AIG Content Reuse Working Group

The Content Reuse Working Group (#digreuse) focuses on developing standards, strategies, and workflows for assessing the use and reuse of digital objects across galleries, libraries, archives, museums, and repositories (GLAMR)

Content Reuse Working Group in 2020

The Content Reuse Working Group is developing the Digital Content Reuse Assessment Framework Toolkit (D-CRAFT). The D-CRAFT project, generously funded by a National Leadership Grants for Libraries (LG-36-19-0036-19) award from the Institute of Museum and Library Services, started in July 2019. 

Working group members are building the toolkit in two phases of work. The first phase includes work on ethical guidelines and recommended practices. 

  • Ethical Guidelines. The Guidelines are intended for practitioners assessing use and reuse of digital cultural heritage artifacts, research outputs and scholarship, and data. These guidelines are meant both to inform practitioners in their decision-making, and to model for users what they can expect from those who steward digital collections. Integral to the creation of this code are user privacy considerations, and a particular focus on concerns and ideas of Black, Indigenous, people of color, people with disabilities, working class, and poor communities. The working group is currently finalizing an Ethical Guidelines for Assessing Reuse draft for public review and comment. 
  • Recommended Practices. Recommended practices will document tools, resources, and existing strategies for assessing various facets of digital object reuse.

Along the way, the working group will develop draft content with D-CRAFT Consultants, solicit feedback from their Advisory Group and GLAMR practitioners, and launch the toolkit.

How can you participate?

We encourage those interested in knowing more about the project to visit the D-CRAFT project site as well as to review a 2019 DLF Blog post Introducing D-CRAFT. Those who have questions or comments should feel free to reach out to Santi Thompson.

AIG Cost Assessment Working Group

We dig into the cost (both time and money) of digitization.

Cost Assessment Working Group in 2020

Building on the development of the Digitization Cost Calculator, the AIG Cost Group is shifting directions to create a more expansive toolkit that will serve a wider range of digitization labs. 

  • Survey. We are seeking information from a variety of digitization stakeholders from across the DLF to inform two new initiatives. The data collected will be used to help create resources for calculating and assessing institutional commitments necessary for successful digitization projects or starting a digitization program.
  • Resource Library. We are looking to assemble a resource library similar to the DLF AIG Metadata Application Profile Clearinghouse Project. The survey data will help us gather some preliminary information about the various organizations successfully using cost assessment tools in their local context. If possible, we will link to or provide samples which may provide a template for other organizations.
  • Cost Worksheets. We seek more data on those interested in cost assessment practices that will give us better insight into how to further develop templates for “cost worksheets” which might be applied at the institutional level and modified for local use. Unlike the cost calculator, these worksheets would be generated from local information to give end-users more contextualized information. 

Digitization costs can be calculated in terms of either/both money and time, which themselves exist in tension with one another. For the purposes of this work, digitization is defined as the practices and procedures associated with reformatting physical objects into a digital form and may include, but are not limited to: materials preparation, digital capture activities, quality assurance, post processing, and metadata creation.

Stay tuned for the survey! And if you’re interested in participating in this cost work, consider joining us at our monthly Zoom meetings. All are welcome!

How can you participate? 

  • Join our meetings! We welcome participation from anyone with an interest in this area. If you join our Digital Library AIG Cost Assessment Google Group you will get notifications and invitations each time we meet. You’re also welcome to drop in on any of our standing meetings which are listed on the DLF Community Calendar; call information is sent the day of the meeting to the DLF Announce Listserv. Questions? Contact co-facilitator Shannon Willis.
  • The group typically meets once a month, and the time commitment is generally less than an hour per week.

AIG Cultural Assessment Working Group (CAWG)

To raise awareness of cultural bias and strive for diversity, equity and inclusivity [1] in digital collection practice to create more inclusive cultures and to mitigate or expose collection bias. Where such practice is lacking, create new frameworks that uphold CAWG values.

CAWG in 2020

The Cultural Assessment Working Group formed in February 2016 to discuss ways by which we may assess our digital collections and their cultural impact. Since its founding, the group has investigated and attempted to surface underlying assumptions within the digital collection creation process and practices to help the community better understand the social structures that both influence our work and result from it. CAWG Task Forces develop or refocus each year to focus on specific projects including the following:

Inclusive Metadata Task Force (IMTF): 

The IMTF is organizing our efforts this year around two main threads:

  • Discussions. Monthly discussions of a specific topic, resource, or reading. Recent discussion topics have included: 
  • Toolkit. Ongoing content gathering for a public-facing “Inclusive Metadata” Toolkit. Our goal for the Toolkit is to create a resource that will support folks who want to begin and foster conversations at their own organizations, to promote inclusive metadata practices. 

Through our monthly readings/discussions, we are increasing our awareness of some of the most issues and challenges across various organizations, and generating ideas for common challenges and relevant sources we want to include in our Toolkit. Along with these external sources, we are also keeping a running list of questions and challenges that Task Force members run into in their work. This list provides us with ideas for future discussions within our group, as well as potential case studies to explore in the Toolkit. 

Digitization Selection Task Force:

The Digitization Selection group is a new group within CAWG this year, created to focus CAWG related efforts on selection as it relates to digitization. The group devoted initial resources to brainstorming possibilities, including discussing digitization and selection criteria. This resulted in numerous opportunities for conversations among group members from a diverse range of professional backgrounds. These conversations lead to several work products, including an initial draft of a visualization and 40 accompanying questions. 

Work Products:

  • Visualization. For its first project this year, the group created a visualization to outline the different workflows involved in digitization, especially as they related to the process of selection. The goal is to create a visual reference point for our future action items. Once the Digitization Workflow Visualization is completed, the group will share it with the rest of CAWG as so many aspects of selection relate to the other working groups.
  • Questions. The group decided to compile 40 questions to accompany the Digitization Workflow Visualization to help illustrate the various aspects of each process of digitization selection. A secondary goal is to gather information about what steps various institutions are taking to make their assessment criteria more inclusive of digitizing collections from traditionally marginalized groups, and to incorporate these findings into the set of questions in order to foster inclusivity when assessing collections for digitization in our own documentation. Towards the end of this year, the group shared the draft questions with the CAWG Inclusive Metadata Task Force for feedback. Next steps include an environmental scan of related publications and digitization assessment policies.

Selection Task Force:

The CAWG Selection Task Force is conducting a survey of national, regional and local arts and humanities organizations whose collections reflect and represent the cultural diversity of their membership. We seek a greater understanding of the practices and policies that organizations use to build community archives or libraries, especially with regards to how materials are selected. The task force is interested in organizations with established collection activities and those that want to begin collecting materials. The goal is to incorporate the survey results into the CAWG Toolkit to be more inclusive of community cultural organizations. In addition, the task force will share the survey results and the toolkit with community organizations to assist their important curation efforts.

How can you participate?

We are keen to connect with individuals and groups within and outside of DLF working on similar topics, so please reach out to collaborate! Our meetings are open and we always welcome new members. We welcome insight from new perspectives even as we finalize deliverables.

  • Join the conversation. Sign up for the CAWG Google Group indicating your interests and drop in on any of our open meetings (details via Google Calendar). We post all updates and meeting details via the Google Group as well.
  • Attend a meeting. CAWG typically meets as a group once a month, and Task Forces meet depending on their need throughout the year, anywhere between once a month and every other week. There are various roles you can have in CAWG included as a Lead Implementer, Facilitator, or Participant in any group.

AIG Metadata Assessment Working Group

The Metadata Assessment Working Group (MWG) supports the development of guidelines, best practices, tools, and workflows relating to the evaluation of metadata used in digital libraries and repositories. This goal is achieved by supporting discussions of metadata assessment and quality control, sharing metadata assessment approaches, and fostering a community of dedicated metadata assessment professionals. 

The MWG often forms sub-groups to work on specific projects (see below).

Metadata Working Group in 2020

Information about past and current work can be found on our website: http://dlfmetadataassessment.github.io/. This year, members of the MWG identified several priorities including the following projects:

  • Benchmarks. A long-term goal for this group is to establish metadata quality benchmarks for organizations, so that they may evaluate their collections according to a common standard. As an initial step, members of the MWG developed and distributed a survey on the ways that organizations evaluate their own metadata. They also published a white paper detailing the survey’s preliminary findings. Information about this project can be found here: http://dlfmetadataassessment.github.io/MetadataBenchmarks
  • Environmental Scan. In 2016, the MWG compiled publications, presentations, tools, and an organizational directory related to metadata assessment. This year, we focused on reviewing collected documentation and determining the best ways to organize and update the following resource: https://dlfmetadataassessment.github.io/EnvironmentalScan
  • Metadata Application Profile Clearinghouse. The Clearinghouse provides a central space where users from various institutions can contribute, compare, and research different application profiles, best practices, mappings, crosswalks, and metadata code. We are currently discussing ways to improve this resource’s maintenance and submission processes. The Clearinghouse can be accessed via the following link: https://dlfmetadataassessment.github.io/MetadataSpecsClearinghouse/
  • Tools Wiki/Repository. The Tools Wiki/Repository is a compiled list of tools that could be used to assess, create, and transform metadata. Most of the work this year has been general maintenance to add information about tools that were not yet on the list and determining the most useful format to display tool descriptions. The link to the Tools Wiki/Repository is provided here: https://github.com/DLFMetadataAssessment/Tools/wiki
  • Twitter Chat. Although not part of the original work plan for the year, the MWG held a Twitter chat in June to informally gather information about how the COVID-19 pandemic has impacted metadata activities. We are considering additional chats, although no additional events are currently planned. The June chat is archived here: https://wakelet.com/wake/CvsOGuOSpOffK8exGqscw

How can you participate?

Anyone with an interest in metadata quality and assessment should consider attending an MWG meeting and joining our discussions and project groups. New members are always welcome, regardless of professional background or technological expertise! 

If you are interested in this working group, join the Metadata Working Group Google Group to receive announcements and agendas for upcoming meetings. Please note that:

  • For 2020, we meet every other Thursday at 1:15 pm Eastern / 12:15 pm Central / 11:15 am Mountain / 10:15 am Pacific. Details on how to connect can be found on our wiki page
  • We also distribute meeting agendas with connection information via the metadata channel in the DLF AIG Slack workspace

Thank you for taking the time to learn more about the DLF Metadata Assessment Working Group. We hope to see you at one of our meetings!

AIG User Experience Working Group

The User Experience Working Group takes a multi-faceted approach to determining the best way to develop guidelines and best practices in user and usability studies related to digital libraries.

User Experience in 2020

The UX group spent 2020 on two projects. 

  • Remote UX. A document to help record information about the experiences of doing UX work remotely as COVID-19 affected the globe. 
  • UX Maturity. We’re working on increasing UX maturity in GLAM organizations.

How can you participate?

  • The User Experience group meets once a month on the last Friday of the month. Members are expected to spend at least two hours a month on group work.
  • To sign up please join the Google Group, post something to introduce yourself, and send an email to Joy M. Perrin so that your name and email can be added to the monthly meeting.


For questions about the larger DLF Assessment Interest Group, contact the DLF AIG co-leaders, Sara Rubinow and Santi Thompson.

Be sure to join the AIG Google Group and sign up for Slack!

This post comes from:

Did you enjoy this post? Please Share!


Related Posts

DLF Digest: April 2024

  A monthly round-up of news, upcoming working group meetings and events, and CLIR program updates from the Digital Library Federation. See all past Digests

Skip to content