This paper proposes SCOPE-MH (Safety Claims Over Preserved Evidence) as a diagnostic complement to existing evaluation infrastructure and argues that evaluation preserving temporal evidence is necessary, not optional, for safety-critical mental health AI deployment.
Multi-agent debate improves LLM reasoning, yet agreement is not evidence of correctness. When agents converge on a wrong answer through social reinforcement, consensus stopping commits that error to automated action with no recourse. We introduce Conformal Social Choice, a post-hoc layer that converts debate outputs in...
Mengdie Flora Wang, Haochen Xie, Guanghui Wang et al.· 0 citations
This work introduces a model-agnostic association rule learning framework that extracts association rules from any conditional probabilistic model over tabular data, and introduces TabProbe, an instantiation of this framework that utilizes TFMs as conditional probability estimators to learn association rules out-of-the...
Erkan Karabulut, Daniel Daza, Paul Groth et al.· arXiv.org· 0 citations
Large language model (LLM) evaluation commonly relies on fixed human-authored benchmarks or centralized human/model judges. Fixed benchmarks can age and enter model training corpora, while centralized judges introduce their own scoring preferences.
We cast LLM evaluation as a multi-agent peer-assessment problem and i...
Yanki Margalit, Nurit Cohen-Inger, Erni Avram et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
The findings reveal that the field has been optimizing for proxy metrics that do not predict deployment performance, and that current personalized alignment methods fail to operationalize preferences into behavioral adaptation under realistic deployment constraints, and finds simple in-context learning (ICL) to be high...
Model merging aims to integrate task-specific abilities from individually fine-tuned models into a single model without extra training. In recent model merging methods, task vector has become a fundamental building block, as it can encapsulate the residual information from finetuning. However, the merged model often su...
Bang An, Yibo Yang, Philip Torr et al.· 0 citations
Designing high-performing heuristics for vehicle routing problems (VRPs) is a complex task that requires both intuition and deep domain knowledge. Large language model (LLM)-based code generation has recently shown promise across many domains, but it still falls short of producing heuristics that rival those crafted by...
Andr\'e Hottung, Federico Berto, Chuanbo Hua et al.· 0 citations
The generalization of vision-language-action (VLA) models heavily relies on diverse training data. However, acquiring large-scale robot manipulation data across varied object appearances is costly and labor-intensive. To address this, we introduce Embodied Manipulation Media Adaptation (EMMA), a framework for augmentin...
Zhehao Dong, Xiaofeng Wang, Zheng Zhu et al.· 0 citations
Bus bunching remains a challenge for urban transit due to stochastic traffic and passenger demand. Traditional solutions rely on multi-agent reinforcement learning (MARL) in loop-line settings, which overlook realistic operations characterized by heterogeneous routes, timetables, fluctuating demand, and varying fleet s...
Integer programming (IP) is central to many combinatorial optimization tasks but remains challenging due to its NP-hard nature. A practical way to improve IP solvers is to manually design inequality families that exploit problem structure. However, this creative process requires deep expertise and has therefore been di...
Milad Yazdani, Mahdi Mostajabdaveh, Samin Aref et al.· 0 citations
This paper frames the evaluation of LLMs in planning as a natural language processing (NLP) task, as LLMs are themselves NLP models, and proposes a pipeline comprising three NLP-based steps that assess plan quality from complementary perspectives, followed by a symbolic planning step to achieve a valid plan to provide...
Ma'ayan Armony, Albert Meroño-Peñuela, G. Canal· arXiv.org· 6 citations
Gene expression analysis holds the key to many biomedical discoveries, yet extracting insights from raw transcriptomic data remains challenging due to the complexity of multiple large, semi-structured files and the need for extensive domain expertise. Current approaches either rely on inflexible workflows that fail in...