Automated Coding for Discourse Analysis: A Case Study of the Böhmermann–Erdoğan Humor Scandal
TL;DR
The findings suggest that automated coding can complement manual discourse analysis by enabling faster, more systematic comparison of media frames in multilingual corpus, thereby contributing to debates on the use of generative artificial intelligence in qualitative and mixed-methods research.
Abstract
Abstract This study examines the effectiveness of artificial intelligence for automated discourse coding in humor scandal research, using the Böhmermann–Erdoğan Humor Scandal (BEHS) as a case study. Humor scandals are controversies triggered by humor that some find amusing while others deem offensive, generating moral outrage, social or political division. Although interpretive approaches have yielded important insights, computational methods remain underused despite the large-scale responses characteristic of digital media environments. BEHS began in 2016 when comedian Jan Böhmermann recited a satirical poem about the Turkish President Recep Tayyip Erdoğan on German television, sparking a controversy that escalated into a diplomatic dispute between Turkey and Germany. To analyze how the scandal is framed across linguistic, national, and ideological media contexts, the study draws on a manually coded corpus of 237 Turkish- and English-language newspaper articles. This corpus comprises 3,476 coded segments and 5,541 code instances. Using this data, the study compares three approaches to discourse coding: label-only zero-shot prompting, codebook-guided zero-shot prompting, and supervised fine-tuning (SFT). The study also compares intralingual and cross-lingual learning transfer from Turkish to English. The results show that SFT substantially outperformed the prompting-only approaches. The best-performing configuration, a fine-tuned gpt-4.1-mini, achieved a micro-F1 of 0.884 for Turkish and 0.789 for English, indicating that automated coding is highly reliable interlingually and moderately effective cross-lingually. Overall, the findings suggest that automated coding can complement manual discourse analysis by enabling faster, more systematic comparison of media frames in multilingual corpus, thereby contributing to debates on the use of generative artificial intelligence in qualitative and mixed-methods research.