Distillation of a Locally Deployable 8B Language Model for Lung Histology Abstraction: A 33-Report Proof-of-Concept Study

Abstract

Background The manual data abstraction process for lung cancer registries is a labor-intensive undertaking that often results in registry lag. While cloud-based Large Language Models (LLMs) exhibit high reasoning acuity, their reliance on third-party APIs poses significant data privacy risks and conflicts with institutional data sovereignty. Purpose The objective of this study is to develop and provide a pilot proof-of-concept validation of a secure, tiered hybrid architecture for the localized four class extraction of lung cancer pathology reports using domain-specific Small Language Models (SLMs). Methods We employed an instruction-based distillation paradigm, transferring clinical reasoning from Gemini 2.0 Flash (Teacher) to a quantized Med42-8B model (Student) alongside a generalist comparator (Llama-3-8B). A six-level WHO Tiered Logic framework based on 2021 standards was implemented as a semantic constraint to stabilize reasoning. The training process was executed for a single epoch on a consumer-grade GPU (NVIDIA RTX 3060) utilizing 2256 MIMIC-IV records. To evaluate real-world clinical feasibility and execution stability, the localized framework was deployed on a broader cohort of 134 independent, de-identified clinical records from Taipei Medical University Hospital (TMUH). From this cohort, a targeted pilot subset of 33 specialized pathology reports underwent a rigorous, double-blind clinical forensic audit by human experts to establish an absolute performance and diagnostic safety benchmark. Results Across the 134-case deployment cohort, the specialized model achieved a 100% JSON syntactic compliance rate without execution failures. Within the audited 33-case clinical subset, the distilled domain-specific Med42-8B model demonstrated a diagnostic accuracy of 90.91% and a macro F1-score of 0.92 (substantial agreement with experts, κ = 0.86), significantly outperforming the general-purpose Llama-3-8B student model (accuracy: 78.79%, macro F1-score: 0.79). Qualitative safety audits on confounding cases (eg, thymoma and metastatic sarcoma) demonstrated that the specialized model successfully adhered to Level-1 gatekeeper logic to prevent clinical safety hazards, whereas the generalist model suffered from critical logical hallucinations. The implementation of a deterministic StopOnProperJSON mechanism effectively resolved issues related to generative looping, ensuring data integrity during local execution. Conclusion This exploratory proof-of-concept study demonstrates the technical feasibility of localized, domain-distilled SLMs as a privacy-preserving foundation for four-class lung histology abstraction. While statistical generalizability remains bounded by the pilot audit cohort, the framework offers an accessible, privacy-compliant methodological blueprint that warrants prospective validation across larger multicenter cohorts.