For this special edition of DLF Contribute, we explore IRUS-USA (Institutional Repository Usage Statistics USA), an experimental collaboration between Jisc and the Digital Library Federation (DLF) at CLIR.
Jo Lambert manages services and projects at Jisc, a UK registered charity that champions the use of digital technologies in education and research. Paul Needham is the Research and Innovation Manager at Kings Norton Library, Cranfield University. Santi Thompson is the Head of Digital Research Services at the University of Houston (UH) Libraries and a co-leader of the DLF Assessment Interest Group (AIG).
Full Bios
Jo Lambert manages services and projects at Jisc, a UK registered charity that champions the use of digital technologies in education and research. Current work includes managing shared analytics services, the Journal Usage Statistics Portal (JUSP), and Institutional Repository Usage Statistics (IRUS-UK) in the UK, as well as a series of projects to provide services or explore use outside the UK. With a background in information services and project management, Jo is interested in working alongside higher education (HE) communities to develop practical, evidence-based shared projects and services that meet community needs and support organisations in their decision making.
Paul Needham is the Research and Innovation Manager at Kings Norton Library, Cranfield University. He is a member of the NISO SUSHI Standing Committee and the COUNTER Executive Committee, and co-chair of the COUNTER Technical Advisory Group. Since 2008, he has mainly worked on projects and initiatives relating to usage statistics based on the COUNTER standard. These include involvement in JUSP (the Jisc Usage Statistics Portal); the new Release 5 of the COUNTER Code of Practice; development of the COUNTER_SUSHI_API; several IRUS services including IRUS-UK, IRUS-CORE, IRUS-OAPEN; and IRUS pilots: IRUS-ANZ and IRUS-USA.
Santi Thompson is the Head of Digital Research Services at the University of Houston (UH) Libraries and a co-leader of the DLF Assessment Interest Group (AIG). Santi publishes on the assessment of digital repository metadata, software, and content reuse. He also currently serves as the principal investigator for the IMLS-funded “Developing a Framework for Measuring Reuse of Digital Objects” grant project and the co-principal investigator for the IMLS-funded “Bridge2Hyku Toolkit: Developing Migration Strategies for Hyku.”
Jo and Paul, could you tell us a bit about IRUS-UK? What has been Jisc’s motivation for developing and investing in a system like this?
Jo and Paul: 15 years ago, institutional repositories (IRs) were the new in-thing. Higher education institutions, just about everywhere, were setting up institutional repositories. Money was being spent, time and effort were being expended . . . but there was no way to reliably demonstrate the usage and impact of those repositories. Sure, statistics were being generated—but everyone was doing it differently, applying their own rules to processing data, and many figures produced were vastly inflated by search engine and robotic usage. We were trying to compare apples and oranges. The statistics lacked credibility.
So that’s why we started IRUS-UK—the first service to enable IRs to expose and share usage statistics based on a global standard—COUNTER. The COUNTER standard is the one that traditional scholarly publishers and aggregators like Elsevier, Springer, EBSCO, etc. all adhere to when producing usage statistics. We all follow the same rules and usage data are filtered to remove robots and double clicks, so the statistics are reliable, trustworthy, authoritative, and comparable.
IRs use IRUS to monitor and benchmark usage of their research against similar organisations in a meaningful way. It provides Jisc with a view of UK repository use to demonstrate the value and impact of IRs. And it provides a UK-wide launch pad for collaborating with other national and international initiatives, projects, and services. We were really interested to hear about DLF AIG work and as our conversations developed our common interests became more and more apparent. A mutual interest in tools to measure impact, develop benchmarks, and share ideas and good practice prompted the collaboration that has since morphed into IRUS-USA.
Santi, as co-chair of the DLF Assessment Interest Group (AIG), what DLF AIG connections and research interest led you to recommending our current IRUS-USA pilot project?
Santi: The JISC-funded Institutional Repository Usage Statistics (IRUS) aggregation project excited me for several reasons. First, I have found difficulty in gaining access to standardized usage statistics for scholarly works repositories. While many systems offer built-in statistics features, they often lack documentation that offers details on how they work, including what they do (and do not) count. With the COUNTER standard acting as the foundation for the aggregation service, IRUS draws upon standardized practices to deliver usage statistics across a shared community, giving managers access to a diverse range of data. The ability to query the usage statistics by formats and benchmark against other member institutions offers repository managers collection development tools often lacking in institutional repository environments.
The work of IRUS also intersects nicely with current and former projects sponsored by the DLF AIG. A former working group, the Web Analytics Working Group, focused a large portion of their efforts on compiling information on various analytic tools and services that could aid in assessing repositories. In 2015 the group published a white paper on the use of Google Analytics in Digital Libraries. The group followed up this work in 2016-2017 by developing an annotated bibliography on how libraries use web analytics to assess their programs, collaborate with other institutions, and make decisions. There work provides a great overview of the world of usage analytics.
Another AIG group, the Content Reuse subgroup of the User Studies Working Group, is currently investigating how best to assess the reuse of digital objects. With funding from IMLS (Developing a Framework for Measuring Reuse of Digital Objects [LG-73-17-0002-17]), the group is aiming to expand upon standardized usage statistics to better understand how users utilize or transform unique materials from library-hosted digital collections. The team believes that leveraging usage statistics, like the kind provided by IRUS, and reuse information will provide practitioners with a richer set of data in which to highlight the value of digital repositories and cultural heritage organizations.
What are all three of you geeking out on? (Or, what is the most interesting thing you’ve learned through this IRUS-USA initiative?)
Santi: My participation in projects like the IRUS-USA pilot program and the Measuring Reuse grant program have me obsessed with better understanding digital library users and reuses. The deep dive that I and my colleagues have taken on who uses digital library materials and for what purposes has allowed me to see how digital libraries are just as much of a “public good” or “public service” as they are a scholarly resource. There are countless anecdotes of how “everyday” people are using digital library objects for a variety of purposes—personal research, genealogy and family history, artistic expression and creation among others. However, I am not sure how well we, as a profession, have embraced the “public good” aspects of digital libraries and think that more attention can be given to the relationship between digital libraries and the “everyday” user. I will continue to collaborate with colleagues to explore this relationship.
Jo and Paul: After several years of developing IRUS within the UK, we’re geeking out on seeing a growing appetite for an international family of services that can interoperate with one another to provide a global picture of IR and OA usage. We developed IRUS-UK by working with universities to understand what they need and then delivering a service to meet that need, so we’re hyped to have a US dimension to IRUS through the IRUS-USA pilot project, and excited about the potential for international measurement and benchmarking. Working with CLIR and DLF AIG folks has given us a greater insight into work in the US right now around use and perceptions of analytics. It’s enabled us to learn from their ideas and approaches and to work collaboratively to develop a pilot service. We’re looking forward to working with our colleagues in the US in the coming months and years.
