A stylometric machine learning framework for authorship attribution of ChatGPT and student generated texts

Abstract

Introduction The surge of generative artificial intelligence (AI) has created major challenges in determining authorship, evaluating student learning, and maintaining academic integrity. Distinguishing AI-generated text from human-generated writing has therefore become increasingly important for authorship verification, educational assessment, and academic honesty. Methods This study proposes a machine learning-based stylometric approach for authorship prediction of short-story adaptations written by non-native German language students and generated by ChatGPT. A corpus of 60 texts was created, comprising 30 student-written adaptations and 30 ChatGPT-generated adaptations based on the same source stories. The proposed framework integrates text preprocessing, stylometric feature extraction, feature selection, and machine learning-based classification. Linguistic and stylometric features included lexical diversity, word and character frequencies, sentence and paragraph length distributions, syntactic complexity, function-word usage, punctuation patterns, readability measures, and semantic consistency. Support Vector Machine (SVM), Random Forest, Decision Tree, and Naïve Bayes classifiers were evaluated. Results The linguistic analysis revealed significant differences between the human-generated and AI-generated text categories. Among the evaluated classifiers, the SVM achieved the best classification performance, with an accuracy of 94.8%, precision of 93.7%, recall of 95.2%, and F1-score of 94.4%. Lexical diversity, sentence complexity, function-word frequency, and readability were identified as the most influential features in authorship prediction. Discussion The findings demonstrate the effectiveness of the proposed stylometric framework for automated authorship attribution and AI-generated text detection. The approach has potential applications in educational assessment, plagiarism detection, forensic linguistics, authorship verification, and the development of intelligent tools for supporting academic integrity in multilingual learning environments.