Skip to content
Book Open access

ConvoDojo: Structured LLM-based Sparring Partners for Difficult Workplace Conversations.

Jul 2026 · International Conference on Conversational User Interfaces · pp. 1-15 · 0 citations · 47 references
Computer Science

TL;DR

This paper showcases the ConvoDojo platform and presents an evaluation of how key conversational user interface design elements, namely, the addition of structured feedback and upfront instructional scaffolding, impact managers’ learning.

Abstract

Large language models (LLMs) often exhibit sycophancy, optimizing for agreement over productive challenge, which severely limits their utility in domains like professional skills training, where growth requires pushback. We introduce, ConvoDojo, a novel conversational AI platform for practicing difficult workplace conversations, engineered not merely as a commercial training application but also as a flexible, instrumented research platform for evaluating conversational AI strategies. ConvoDojo repurposes LLMs as structured sparring partners to support skill development in difficult workplace conversations (e.g., performance feedback, conflict resolution), addressing the reported managerial tendency to avoid them. This paper showcases the platform and presents an evaluation of how key conversational user interface (CUI) design elements, namely, the addition of structured feedback and upfront instructional scaffolding, impact managers’ learning. Results show that ConvoDojo is highly engaging and promotes user reflection. We demonstrate how theory-informed dialogue and adaptive pushback can transform an LLM into an effective, measurable tool for complex communication skills development.

Read PDF

Similar papers

MonitrLLM: A Community-Centered Evaluation Infrastructure for Large Language Models

MonitrLLM is introduced, open-source infrastructure for community-centered LLM evaluations that links full conversation transcripts to user-reported task intent and outcome assessments, treating all three as primary evaluative signals rather than optional metadata.

Victor Ojewale, Ro Encarnación, S. Venkatasubramanian et al. · 0 citations
Conference Open access Aug 2026

Nobody Talks About the Prompt Club: Evaluating Diverse Persona Generation Through System Prompts and Guidelines

Personas are widely used to represent users' motivations, goals, and needs, yet the perspectives of underrepresented groups are rarely addressed in human-centered design. Large Language Models (LLMs) promise to generate personas from vast datasets, but achieving genuine diversity and accurate representation requires de...

C. Lazik, Charlotte Kauter, Isabella Graßl et al. · 0 citations
Open access Jul 2026

Xbot: a GPT-based chatbot with transparent and empathetic behaviour

Experimental comparisons with GPT-4o vanilla across three roles, evaluated through an ablation study and a multi-evaluator panel combining LLM-based and human judges, consistently rank XBot as the best performing system, demonstrating superior empathy, role stability and conversational depth, while GPT-4o vanilla exhib...

Luciano Caroprese, Ester Zumpano, M. Aracne et al. · 0 citations
Jul 2026

Beyond Direct Answering: Aligning Educational LLMs as Socratic Guides via Heuristic Reinforcement Learning

HeuristicEdu is presented, a two-phase pipeline that aligns Qwen2.5-7B toward Socratic tutoring via supervised warm-up and Group Relative Policy Optimization (GRPO), and Scaffolding Effectiveness and Conversation Depth are introduced to evaluate outcomes beyond surface fluency.

Xiaokun Wang, Si-Yu Song, Wentao Liu et al. · 0 citations
Review Open access Aug 2026

Beyond Neutrality: Comparative Study of ChatGPT-Produced Recommendation Letters

It is claimed that LLMs are the latest AI-generated product utilized in academic discourses, enriching the capacity of the user and could be minimized by careful attention, capacity, tactful utilization, and knowledge of the user.

A. Sharma · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.