ChatGPT vs. Gemini: Which model generates more comprehensible texts and more solvable tasks?

Abstract

Large language models (LLMs), which are computer programs designed to understand and generate human language, are increasingly common in foreign language teaching. These models can reduce the time and effort teachers need to put in, and they help students learn a foreign language on their own, without frustration or fear of embarrassment. This study aims to determine which LLM enables students to better understand texts and solve accompanying tasks more successfully. Texts and tasks generated by ChatGPT and Gemini are compared. The LLMs were asked to generate texts and tasks on two recurring topics in Croatian German language textbooks: sports and regional studies. Primary school students at a lower secondary educational level learning German as a foreign language participated in the study. The results show that the students had more correct answers to the tasks generated by ChatGPT than to those generated by Gemini. ChatGPT responds more quickly and directly to prompts, thereby generating teaching materials without verbose explanations.      Keywords: ChatGPT; Gemini; Large language models; tasks