Skip to content

Human vs. Generative AI in Qualitative Analysis: Assessing ChatGPT 5.1 in Identifying Evaluative Judgment in Conflict Media Discourse

Jul 2026 · Journal of Media Research · Vol 19, pp. 53-71 · 0 citations

TL;DR

The initial results of a comparative analysis designed to evaluate the performance of human coding and GenAI (ChatGPT 5.1) coding in qualitative content analysis reveal both opportunities and limitations in the use of AI for qualitative research.

Abstract

This exploratory study presents the initial results of a comparative analysis designed to evaluate the performance of human coding and GenAI (ChatGPT 5.1) coding in qualitative content analysis. The analysis focuses on six selected articles from Italian media outlets (La Repubblica, Il Corriere della Sera, Il Giornale) covering the Russia–Ukraine conflict from 2022 onward. These six articles were selected from a broader corpus of 180 publications, which served as the basis for the sampling procedure. The findings reveal both opportunities and limitations in the use of AI for qualitative research. On the one hand, ChatGPT provided additional capacity for identifying linguistic units that had been overlooked by the human coder. On the other hand, ChatGPT showed limitations related to its constrained linguistic capabilities, which prevented it from fully capturing complex linguistic figures associated with judgment values. This research offers methodological insights for communication scholars regarding the responsible and effective integration of GenAI into qualitative media studies. Future research will extend the qualitative analysis to the broader corpus, with attention to framing strategies, manipulative practices, and objectivity markers. This continued comparative approach will allow for a more detailed mapping of similarities and differences in coding tendencies between human coding and AI, as well as for the detection of ideological patterns in Italian media coverage of the Russia-Ukraine conflict.

View source

Similar papers

Sep 2026

Reading Between the Lines: Evaluating GenAI's Detection of Attitudinal Meaning in News Discourse

This study investigates the potential and limitations of ChatGPT in identifying attitudinal resources in news discourse. Fifteen news articles were analyzed by ChatGPT, and the results generated by it were compared with manual annotations obtained in a previous study. The findings show that ChatGPT is capable of identi...

Hong Wu · 0 citations
Review Open access Sep 2026

A Systemic Functional Linguistic Analysis of ChatGPT’s Academic Writing Responses: Metafunctional Patterns and Implications for Writing Pedagogy

Generative artificial intelligence (AI), particularly ChatGPT, is having a significant impact on how academic writing is produced, taught and assessed in higher education. This study uses Systemic Functional Linguistics (SFL) to analyze how ChatGPT constructs meaning when it responds to academic writing prompts. Drawin...

Taimur Khan, Lei Feng · 0 citations
Review Open access Sep 2026

AI-assisted speaking in ELT: a bibliometric mapping of research gaps and a future research agenda (2023–2026)

The rapid integration of artificial intelligence (AI) into English Language Teaching (ELT) has generated a fragmented research landscape regarding AI-assisted speaking instruction. While individual studies report promising outcomes for chatbots and automatic speech recognition (ASR) systems, the field lacks a systemati...

Mustafa Özdere · 0 citations
Open access Oct 2026

Translating ambiguity: A comparative evaluation of AI Chatbots in rendering Arabic full homonyms into English

The study concludes that AI systems require substantially more training, particularly in Arabic-language contexts, and recommended further research on homonym translation, focusing on different contexts as well as the rendition of homonyms outside of pairings.

A. Al-Momani, M. Al-Taher, Sultan Alali · 0 citations
#artificial intelligence Review Open access Nov 2026

Exploring ChatGPT-Assisted Qualitative Data: Comparing Human and AI-Assisted Coding Using Braun and Clarke’s Thematic Analysis

While AI tools like ChatGPT show potential for supporting certain aspects of qualitative analysis, this study highlights key limitations in their ability to fully replicate the complexity and contextual understanding that human-led analysis provides.

Laura Hagan, Alla Chavarga · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.