Finetuning Pre‐Trained Language Models for Automatic Assessment of English‐Chinese Interpreting

Abstract

Recent years have seen growing interest in leveraging pre‐trained language models (PLMs) for automatic assessment of spoken‐language performance. Contributing to this emerging line of work, the present study offers the first attempt to finetune two PLMs, GTE‐multilingual and mmBERT, for assessing English‐Chinese interpreting, using over 1700 human‐scored renditions rated on a four‐band, eight‐point analytic rubric. The models were trained using five‐fold cross‐validation to perform band‐level classification and point‐based score prediction, and were evaluated on a held‐out test set. Performance was assessed using multiple metrics, including Accuracy, Quadratic Weighted Kappa, Mean Absolute Error, and Pearson's correlation. We further conducted ablation studies to examine the contributions of reference interpretations and analytic rubrics to model performance, and initially explored whether interpreting direction affects model scoring behavior. Key findings indicate that: (a) although mmBERT consistently outperformed GTE‐multilingual in score prediction, both models struggled to assess fluency from transcripts alone; (b) rubrics and reference interpretation may offer potentially useful but conditional forms of supervision during model finetuning; and (c) systematic performance differences between the interpreting directions were observed, particularly for fluency. Together, these results highlight the need for construct‐sensitive, supervision‐informed, and direction‐aware automatic scoring approaches for assessment of interlingual interpreting.