A Turkish Biology Dataset and LLM Model

Abstract

General-purpose, low-parameterized language models often produce imprecise or shallow explanations when handling specialized scientific subjects, particularly in non-English contexts like Turkish, where training data is limited. In this paper, an original Turkish biology dataset within the scope of the high school curriculum has been developed; and subsequently, large natural language model studies conducted on this dataset are presented. To synthesize high-quality question-answer pairs, video transcripts and written content were first scraped from public Khan Academy resources. In the two-stage synthesis strategy, by using the GPT-4 API, firstly questions depending on the content and then answers were generated. Data diversity was ensured by producing more than one answer for a single question. The Gemma-3 1B model was fine-tuned on this specialized dataset using the Low-Rank Adaptation (LoRA) method. To measure model performance, LLM-as-a-judge, standard n-gram metrics such as ROUGE, and human evaluation were used. The best-performing configuration, Gemma-3 1B trained on standard-length answers, achieved superior results across all dimensions, reaching M-Prometheus scores of 3.8 for coherence and 3.0 for both correctness and completeness. Additionally, it has been shown that LLM-as-a-judge metrics are closer to human evaluation compared to the ROUGE metric.