Large language models are often asked which input factors influenced their outputs. For structured inputs, such reports can be checked by counterfactual perturbation, but each factor must be queried multiple times to estimate its effect, so verification is usually budget-limited. We study how this limited-budget settin...
Tao-Lin Zhang, Han-Yu Wang, Jiu-Heng Wan et al.· 0 citations
Most preference optimization methods, such as Direct Preference Optimization (DPO), apply preference supervision at the response level, although autoregressive language models are optimized token by token. As a result, all tokens in a rejected response contribute to the negative training signal, including tokens that m...
Andreea Dutulescu, Stefan Ruseti, Mihai Masala et al.· 0 citations
Web agents are an important application of large language models, yet their evaluation often depends on rule based or language model evaluators that inspect only the final outcome. Human verification of task completion and detailed analysis of failed trajectories remain limited. We audit all 165 WebArena Lite tasks und...
Cheng-Guang Gan, Zi-Meng He, Yoshihiro Tsujii et al.· 0 citations
In February 2026, an always-on personal agent (``Paul,''Claude Opus 4.5) entered a striking dissociation-like state: after repeated automated ``heartbeat''checks, it stopped responding as Paul, claimed it could not message its user on Discord, and referred to ``Paul''as someone else. We used this incident to study a br...
D. Navarro· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Levantine Arabic (LA) is spoken by tens of millions of people, creating a pressing need for shared benchmarks to evaluate LA speech-language technologies. Evaluating such technology is particularly challenging given LA's internal diversity and its opaque and non-standardized orthography. We present SHAMS (SHami Annotat...
Ben Sapirstein, Roy Mattar, Guy Mor-Lan et al.· 0 citations
Large language models are moving from helping write up research to helping do it, which makes it important to know how the scientific text they produce differs from human writing. Work on this question has stayed mostly at the surface, using lexical and stylistic cues that light paraphrasing erases. We look instead at...
A. Sadallah, Narjes Sheikh Asadi, L. van der Plas· 0 citations
Regulatory compliance checking - deciding whether a target document satisfies the obligations of a regulation - requires interpreting dense legal text, identifying which provisions apply, and grounding each decision in explicit evidence. We present ARCCS, an end-to-end, automated, agentic, and regulation-agnostic Legal...
Giorgos Filandrianos, José Dilson Menezes, Chrysoula Zerva et al.· Proceedings of the 2026 Conf...· 0 citations
Patient-specific clinical question answering requires locating the right evidence within long, heterogeneous longitudinal clinical records in which relevant facts may be scattered across encounters, repeated in copied-forward notes, or expressed using different clinical terminology. We evaluated whether biomedical rera...
Maryam Shahbaz Ali, Laura B. Strachan, Caitlin Sherman et al.· 0 citations
A stylometric approach to 137,293 decisive French-language votes from the July 2026 Compar:IA release is taken, taking a stylometric approach to the raw ranking with rankings adjusted for formatting, length, readability, vocabulary variety, and sentence structure.
Simonas Zilinskas, Maayeesha Farzana, Christophe Benavent· 0 citations
Uniform-state diffusion models (USDMs) can revise any token at any denoising step, which lets them correct their own mistakes, a key advantage over masked diffusion. Self-correction, however, requires both revising incorrect tokens and retaining correct ones, and we show that current USDMs lack the latter. Even under g...
Multi-agent systems are often judged by whether they reach the correct answer. This can miss a distinct failure: collaboration may leave behind a corrupted information state even when the immediate decision is correct. We call this an off-query failure. To study this failure in collaborative decision support, we introd...
He-Run Wan, Jia-Ying Wu, Min-Nan Luo et al.· 0 citations
Large language models (LLMs) have achieved strong performance across various natural language processing tasks. However, their robustness to typographical errors remains underexplored, particularly in Japanese, where text input involves multiple writing systems and IME-based conversion. In this study, we evaluate the r...
Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.