Enhancing Small Model Information Extraction with Few-Shot Data: A Dynamic Teaching Approach

Abstract

Extracting structured information from massive unstructured texts is a key task for intelligent analysis and knowledge construction, but faces two challenges in low-annotation-resource scenarios: the high cost of obtaining high-quality labeled data and the prevalence of domain-specific terminology and complex semantic structures. Existing methods still face several challenges, including static knowledge transfer, insufficient modeling of learner capability differences, and limited consideration of sample difficulty during teaching strategies. This paper proposes the Dynamic Loop Teaching Method (DLTM), aiming to enhance the information extraction capability of small language models under extreme few-shot conditions. DLTM constructs three identical small models with differentiated initial capacities. Through multi-round dynamic mutual teaching, they adaptively assume asymmetric roles: the relatively stronger model acts as the Optimizer, providing pseudo-supervision signals; the intermediate model acts as the Questioner, generating targeted samples; and the weaker model acts as the Respondent, being fine-tuned. Key components include dynamic role assignment, validation set expansion with rollback protection, and adaptive difficulty curriculum learning.Experiments are conducted on a publicly available Chinese event extraction dataset. The results show that DLTM enhances the ability of small models to extract complex semantic structures and achieves competitive performance compared with representative baselines. In particular, DLTM achieves 17%–25% relative improvement on coreference resolution under corresponding settings compared with ODIE and ADELIE. Further analysis indicates that while DLTM improves semantic-dependent extraction tasks, knowledge-intensive fields remain challenging under extremely sparse supervision.