From newspapers to classrooms: Adapting an annotation scheme and automatic classifiers to mixed-quality argumentative school essays
TL;DR
This work shows that a common approach to representing argument structure as trees is not sufficient to capture the constellations of argument components in student essays, and proposes a suitable extension of that scheme, which is applied to an initial corpus of 50 essays.
Abstract
In argument mining research, educational texts such as student essays have been a popular target genre from the beginning. The annotated corpora available so far, however, focus on essays written by older (or adult) students with relatively high proficiency levels. In our work, we expand the range of texts to German essays by school students in grade 9, which display very different qualities on all levels of analysis. We show that a common approach to representing argument structure as trees is not sufficient to capture the constellations of argument components in those essays, and we propose a suitable extension of that scheme, which we applied to an initial corpus of 50 essays. Furthermore, we conducted experiments with large language models on a fine-grained version of the argument component type classification task and show that medium-sized open-source models can achieve promising classification results using only two labeled essays in a few-shot prompting approach.