2026· International Conference on Language Resources and Evaluation· pp. 6880-6892· 1 citation· 28 references
Computer Science
TL;DR
This work introduces OIR annotations for the NOXI corpus, achieving considerable reliability and evaluating whether LLMs can reliably annotate OIR sequences using structured Chain-of-Thought prompting and comparative analysis across two corpora, finding weak alignment between LLMs and human annotations, particularly in recognizing trouble signaling.
Abstract
Addressing the scarcity of annotated data for Other-Initiated Repair (OIR), when recipients interrupt conversation progressivity to signal trouble, prompting speakers to provide repair, this work introduces OIR annotations for the NOXI corpus, achieving considerable reliability. We evaluate whether LLMs can reliably annotate OIR sequences using structured Chain-of-Thought prompting and conduct comparative analysis across two corpora: NOXI (natural dialogue) and CABB-S (Dutch, task-oriented), finding weak alignment between LLMs and human annotations, particularly in recognizing trouble signaling. Analyzing human-LLM disagreement using the LLM-generated explanations revealed limitations: models rely on lexical patterns rather than conversational context, construct reasonable-sounding but misleading narratives, highlighting crucial limitations for both automated annotation of complex interactional phenomena.
We propose a methodology for investigating the differences between natural and synthetic language in a challenging setting: humorous improvisational dialogue. Our experimental setting compares human-produced comedy improvisations with the dialogues generated by LLMs in a multi-agent environment that we designed. The an...
Lizeth Palacios Patiño, Ximena Gutierrez-Vasques, Daniel Rodriguez· 0 citations
Contact-center dialogue systems require both accurate transcripts and reliable recognition of business-critical entities. We introduce a hybrid keyword and context projector (CP) approach for LLM-based ASR. Rather than appending raw dialogue history, CP encodes previous turns into compact contextual tokens, complemente...
Research for interactive LLMs from retaining more context toward preserving uncertainty is redirected from retaining more context toward preserving uncertainty, to motivate uncertainty-preserving state management.
Jian-Zhe Lin, Xiao-Lin Li, Fei Wang et al.· arXiv.org· 0 citations
Conversational explainable AI (ConvXAI) agents often use GUS-style architectures with separate modules for intent recognition, dialogue management, and template-based response generation. While faithful to the underlying XAI methods, these pipelines are hard to adapt and often struggle with context-dependent follow-up...
Van Bach Nguyen, Qianli Wang, Nils Feldhus et al.· Conference on Empirical Meth...· 0 citations
Long-term conversational question answering can require several separated dialogue turns that support an answer only together. We introduce EviCover, a retrieval approach designed to retain this complementary evidence. It converts questions into observable evidence requirements, checks requirement coverage using retrie...
Effective summarization of interrogative dialogues is a critical task in forensic and investigative settings, requiring high factual accuracy, coherence, and attribute-specific relevance. In this work, we introduce CASPER, a novel Chain-of-Thought Attribute-Specific Prompting for Evaluative Summarization framework that...