This work presents the open-source benchmark framework ParliamentBench based on the game Secret Hitler to evaluate LLMs in scenarios that require deception, persuasion, and reasoning under information asymmetry, and introduces three novel metrics that isolate social deduction, reasoning, and deceptive consistency.
Niklas Bauer, Lars Benedikt Kaesberg, Akiko Aizawa et al.· arXiv.org· 0 citations
It is found that visual presentation is a central factor that determines whether VLM benchmarks measure grounded perception, downstream reasoning, or a mixture of both, and gains are closely tied to reductions in grounding-related errors, while rule reasoning remains comparatively challenging.
Lars Benedikt Kaesberg, Tianyu Yang, F. Wunderlich et al.· 0 citations
This thesis introduces the Multi-Agent LLM (MALLM) framework, which implements and evaluates various decision protocols, namely voting, consensus, and judge decision mechanisms, to simulate multi-agent discussions for conversational task solving and indicates that consensus protocols excel in knowledge-intensive domain...
Lars Benedikt Kaesberg· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.