Piloting the Integration of AI-Driven Detection Methods to Counter Disinformation in Organizational Processes

Abstract

With the rise of generative AI, the increasing threat of automatically generated uncivil content (including misinformation for information warfare up to cyber-bullying purposes) makes the protection of open online discourse even more pressing than before. In the implementation of measures for countering these threats, the different intervention objectives of stakeholders, their workflows, and IT support need to be considered. Stakeholders include online social network moderators, journalists, fact-checkers, social listeners, as well as authorities and organizations with safety- and security-related tasks. Given the sheer volume of online social network content, automated or community-based solutions are required to detect (automated) misinformation. However, detection solutions are predominantly message-focused and target end-users, leaving experts without systematic, large-scale perspectives on coordinated disinformation campaigns to guide countermeasures. To bridge this gap, we conduct an expert-centered study on the integration of detection methods proposed by the research community. Our contributions are threefold: (a) We adopt disinformation features from a previous study and draw connections to literature on detection methods; (b) semi-structured interviews yield vignettes that expose the spectrum of goals, constraints, tools, and decision-making processes employed by experts, informing requirements for method integration; (c) we design a demonstrator that showcases representative methods to uncover unexplored concepts, probe affordances and limitations in context, and evaluate conceptual fit during the interviews. Together, these steps bridge the gap between data- and AI-driven detection techniques from research and the practical needs of diverse stakeholders confronting targeted and large-scale disinformation.