Understanding academic library virtual reference at scale: a multi-institutional content analysis of chat transcripts

Abstract

This study aims to examine the types of questions and requests submitted through academic library chat reference services, how transaction types and complexity vary across institutions and which interactions may be appropriate for automated or artificial intelligence (AI)-assisted response. This study employs qualitative content analysis of 34,977 anonymized chat transcripts (2021–2024) from five US R1 universities. A hybrid stratified random sample (n = 1,746) was manually coded using a collaboratively developed 42-code codebook derived through deductive and inductive methods. Four coders applied up to three codes per transcript, with two rounds of intercoder reliability testing (Krippendorff's α, Fleiss' κ and Cohen's κ) to establish codebook stability. Transaction complexity was operationalized using tertiles based on interaction duration and word count (simple, moderate, complex). Descriptive statistics were used to analyze code frequencies, complexity distributions and cross-institutional variation. A small set of transaction types account for most interactions. Three categories, finding known journal articles, identifying relevant sources and interlibrary loan, appear consistently across all institutions and are predominantly complex. Overall, 38% of interactions were classified as complex, indicating substantial demand for high-level research support. Institutional variation was observed, particularly in access-services transactions. No large-scale, multi-institutional, human-coded analysis of academic library chat transactions using a granular transaction taxonomy currently exists in the published literature. This manuscript introduces a replicable transaction taxonomy and complexity framework to inform evidence-based service design and AI integration.