Same meaning, different words: Automated item paraphrasing in ESP reading assessment

Abstract

Within English for Specific Purposes (ESP) assessment, reading tasks represent the linguistic and discourse demands of academic and professional communication. Maintaining parallel assessment materials while preserving the intended meaning presents an ongoing challenge, particularly in large-scale testing contexts where repeated test administrations can lead to item exposure. This study examines the feasibility of automated item paraphrasing for generating linguistically varied versions of reading assessment materials. A transformer-based Text-to-Text Transfer Transformer (T5) model (Raffel et al., 2020) generated reformulated versions of 16 IELTS reading passages and associated comprehension questions drawn from Academic and General Training materials. The original and paraphrased texts were evaluated using surface-level and meaning-level similarity metrics to assess linguistic variation and semantic preservation. Results showed low to moderate surface similarity alongside consistently high semantic alignment across passages and questions, indicating lexical and structural variation while preserving informational content. These results provide preliminary evidence that AI-assisted paraphrasing may support the development of linguistically diverse ESP assessment materials with high semantic alignment. However, the absence of expert review and test-taker performance data limits conclusions regarding content validity and operational equivalence. The study contributes to emerging research on natural language processing technologies for sustainable item pool management in ESP assessment.