Institutional Books - Visual Elements: An open-source pipeline for extracting, classifying, deduplicating, and captioning visual elements from digital book collections

Institutional Books - Enriched Text: A customizable multilingual open-source pipeline for denoising, deduplicating, and annotating OCR text at scale

Released in 2025, Institutional Books: Harvard Library (IB-HL) is a collection of 983,004 volumes (242B o200k_base tokens), originally digitized through Harvard Library's participation in the Google Books Library project. As researchers and developers have begun to use IB-HL, a tension has emerged between standard large-scale preprocessing practices and the goals of careful information stewardship. Many existing pipelines optimize for web text: as a result, they tend to aggressively filter, deduplicate, restrict by language, and sometimes discard meaningful metadata. Meanwhile, researchers seeking to use IB-HL duplicate effort while performing similar processing and analysis. We describe an approach that we call Enriched Text. Instead of producing a single'complete'stream of tokens, we normalize the text while preserving metadata through annotations. We separate endmatter, detect per-paragraph language, identify clusters of duplicate paragraphs, and compute per-paragraph bits-per-byte scores. We provide this information through HTML-like annotations layered on top of the text. By parsing these annotations, users can tailor the output to their own needs instead of accepting a global editorial decision on content. The pipeline applies to all $\approx$250 languages in the collection. This report describes this project's goals, implementation, and design rationale. The release includes IB-HL-ET (an enriched-text version of IB-HL containing 217B o200k_base tokens across 983,003 volumes, organized into 1.39B annotated subtopic paragraphs) and the pipeline that produced it. These serve to make the collection easier for machines to parse and for humans to study.

Leveraging computer vision and natural language processing for efficient metadata extraction in digitized books

High-quality book metadata improves digital libraries by supporting cataloging, searchability and automated classification. However, metadata generation from scanned books remains challenging due to limited annotated data sets and optical character recognition (OCR) limitations. This study aims to propose a deep learning-based framework for automatic metadata extraction from digitized books by integrating computer vision and natural language processing (NLP). The framework uses MobileNetV2 to classify title pages, table of contents (ToC) pages and content pages. EfficientDet detects metadata-related regions, such as titles and author information. OCR extracts text from these regions, followed by named entity recognition (NER) and regular expressions to refine the extracted metadata. The framework was evaluated using a custom data set of 188 books published between 1800 and 2021, comprising 857 annotated pages. The page classification model achieved 97.16% accuracy. For object detection, the model obtained average precision (AP) and average recall (AR) scores of 71.7% and 42.0% for title pages, 55.1% and 33.4% for ToC pages and 87.9% and 54.4% for content pages, respectively. This study contributes a scalable framework that combines page classification, object detection, OCR, NER and rule-based refinement to reduce manual metadata extraction in digital library management. Future research can expand multilingual data sets, improve robustness to OCR noise and explore end-to-end learning for richer bibliographic metadata extraction.

Automated Indexing of Historical Postcards: An End-to-End Approach Combining Image and Text Analysis

Historical postcards are of great interest to different researchers and especially historians. Preserving these documents for future generations is important. Indeed, they represent valuable cultural heritage documents combining visual imagery with textual information, but manual indexing of large digitized collections is prohibitively expensive in time and resources. This paper presents an end-to-end approach for automated historical postcard indexing that integrates computer vision and natural language processing techniques. Our modular pipeline employs fine-tuned YOLO models to detect text types and postal markings, EasyOCR for text transcription, CamemBERT for named entity recognition enabling automatic geolocation, and DINOv2 models for visual content classification across ten semantic topics. Support Vector Machines predict keywords for images lacking textual descriptions using DINOv2 image embeddings, while complex postal markings are segmented and transcribed. We demonstrate our approach on the recto (illustrated front side) of 4,293 digitized postcards from the Grand Est region of France (1899-1930). Our results show that, among the latest YOLO models, fine-tuned YOLOv12x achieves 0.83 weighted average precision for detection. Fine-tuned DINOv2 attains 0.91 weighted average precision for image classification, while CLIP and other classification models do not exceed 0.83. Finally, named entity recognition successfully geolocates 52% of postcards. In fully automatic operation, the complete pipeline assigns at least a coarse topic to almost every postcard and at least one richer field (a keyword, a geolocation, or a date) to about 80% of them, leaving fewer than 1% essentially un-indexed. Our work contributes to digital humanities by demonstrating effective integration of multiple AI techniques for automated heritage document analysis and provides a publicly available dataset for future research.

CroQS: Cross-modal Query Suggestion for Text-to-Image Retrieval in Image Archives

The novel task of cross-modal query suggestion is introduced, which interactively guides users by suggesting textual refinements based on visual clusters identified in the retrieval results, and the creation of CroQS, a benchmark dataset comprising 50 diverse queries and 295 semantic clusters in generic domain.

Beyond Archives: Designing a Segmentation and ATR Workflow for Mid-Twentieth-Century Italian Typescripts in a Relational Digital Library

An open, documented workflow for layout segmentation and Automatic Text Recognition of mid-twentieth-century typescripts from the Prefecture of Naples for TEI encoding and named-entity extraction is presented.

Digital Approaches to Handwritten Text Recognition in Complex Art-Historical Archives: Towards a Robust Text Model for the Giovanni Battista Cavalcaselle Manuscripts