The Algorithmic Challenges and Opportunities in Tracing Meaning at Scale in Large Historical Document Collections

Abstract

The capabilities of modern language models are expanding the questions that computational history can pose. In this position paper we argue for a research program organized as a progression of increasingly semantic forms of text reuse, moving from surface-level lexical matching, through cross-lingual translation mining, toward the indirect forms of engagement through which ideas are actually transformed. We briefly review the state of the art at each step of this progression, and identify the open frontier, the least direct kinds of engagement. We argue that recognizing such indirect text reuse requires encoder models that move beyond paraphrase, and that historians should contribute to the development of such models. The research program is a case study in fruitful two-directional collaboration: AI enables new history research, whose demands in turn can shape new AI.