What the Corpus Does Not Say: Reply structure, time and coverage in YouTube comment data

Abstract

What linguists and other researchers should check before treating comments as conversation. Anyone planning a research project with YouTube comments meets the same technical hurdles: which tool to collect with, which fields can be trusted, and how to tell whether a corpus is complete. The hurdles are easy to miss when collection and analysis are done with the help of an AI assistant, because the code runs, the files look complete and nothing signals that a field is not what its name says. This paper gathers, in one place, what can go wrong on the way from a YouTube comment section to an analysable corpus, how to detect each problem in minutes, and what to do about it. It is written for corpus linguists, conversation analysts and CMC researchers who read comments as conversation — who answers whom, how address is marked, what follows what, and when. It can be used as a reference document at the start of a project: read before collecting, run as a checklist after collecting, and supplied as a source text to an AI assistant that is helping to build or analyse a comment corpus. What it offers, measured on 210 extractions of 133 videos (480,925 comments, collected with yt-dlp and the YouTube Data API v3): 1. Which source to use for what. The official API files every reply under the first comment of its thread; on the same videos 42.3% and 48.5% of replies receive a different parent than in interface-level extraction. Recommendation: structure from interface-level extraction, time and coverage from the API, both parent fields kept apart. 2. Where reply structure stops being reliable. Interface-level extraction returns replies only to depth 3. At that level the recorded parent and the @-name disagree in 45.8% of cases, one level up in 3 of 13,407. Recommendation: record depth on every edge and treat the deepest level as censored. 3. What the @-name is. In nested replies it is inserted by the interface in about nine of ten cases, so it records a button press before it records a choice of address form. Invisible characters hide 3–4% of markers from a standard pattern; the API glues the marker to the following text in 15.7% of cases. A documented matching rule is given. 4. Why yt-dlp timestamps cannot be used as time. They are back-calculated from labels such as "1 year ago"; in one extraction 182,784 comments share two values, and median error against the API reaches 358 days. 5. How corpora lose comments without trace: replies switched off by one setting (258 videos, 89,074 comments in the author's own collections), runs blocked halfway, a 2,500-comment ceiling, replies orphaned after deletion. 6. Why careful hand-coding is not enough. A coding pass specified in advance confirmed 78 of 100 cases and still supported a wrong conclusion, because the cause lay in tree position. One grouped count would have shown it. Ten diagnostics are collected in Appendix A, each a line or two of code. Applied to the author's next corpus project, they already found a gap in its reporting: the deposit did not show that every requested API page had been retrieved. The paper is also an honest record of AI-assisted research by a non-specialist. It documents how the corpora were built — on a phone, with notebooks written by a language model — the adversarial reviews, the correspondence with the developer of an independent tool, and 25 corrections of the author's own earlier claims. It began as a by-product: a linguistic question about address in YouTube replies that did not survive its own checks, because the pattern turned out to come from the platform rather than from the speakers. No figure is newly computed. All are carried over from the component reports, which this edition supersedes as the reference text and where scripts and tables are deposited: - Six Methodological Pitfalls in YouTube Comment Extraction, v2.7.1: https://doi.org/10.5281/zenodo.22836737 - Flattened Conversations, v3.8.1: https://doi.org/10.5281/zenodo.22117940 - Reconstructed Time, v1.2: https://doi.org/10.5281/zenodo.22214602 Related studies cited: - Two Pointers, One Recipient, v1.1: https://doi.org/10.5281/zenodo.22254594 - Storm Names as Comparative Benchmarks, v3.4: https://doi.org/10.5281/zenodo.22431589 The literature review is not exhaustive. If relevant work has been overlooked, I apologise and would be grateful to be told; suggestions, corrections and refutations are welcome and will be taken up in a later version. Not peer reviewed.