Combee is proposed, a novel framework to scale parallel prompt learning for self-improving agents that speeds up learning and enables running many agents in parallel while learning from their aggregate traces without quality degradation.
Frontier AI systems are increasingly capable and deployed in high-stakes multi-agent environments. However, existing AI safety benchmarks largely evaluate single agents, leaving multi-agent risks such as coordination failure and conflict poorly understood. We introduce GT-HarmBench, a benchmark of 1,535 high-stakes sce...
FlyAOC is presented to evaluate AI agents on end-to-end agentic ontology curation from scientific literature and reveals system-level failure modes that model-only evaluations do not capture.
Reinforcement learning (RL) has become a widely used paradigm for improving the reasoning abilities of large language models (LLMs) and Vision-language models (VLMs). Sparse binary outcome rewards, however, score only final correctness and cannot identify which intermediate steps contributed to it; in multimodal tasks,...
Xiangwei Wang, Wei Wang, Ken Chen et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
This study reveals that the Deepseek-R1-Distill series models are subject to significant fluctuations caused by various factors, making their claimed performance improvements difficult to reproduce reliably.
Yong-Fu Zhu, Lin Sun, Jinzhu Wu et al.· arXiv.org· 4 citations· ⚡2
SkillFlow demonstrates that framing skill acquisition as an information retrieval task is an effective strategy, and that the practical impact of skill-augmented agents hinges on corpus coverage and skill quality, particularly the density of runnable code and bundled artifacts.
A roundtrip benchmark that scores code descriptions by whether code regenerated from them passes the original tests is introduced, and it is shown that completeness, not length, drives a description's fidelity.
This work presents the first systematic evaluation of cultural awareness in LLMs for Haitian Creole, a language spoken by millions but severely underrepresented in digital resources, using a benchmark of culturally salient prompts curated by native speakers in a text infilling setting.
LLMs increasingly act as purchasing agents, which makes the LLM, not the user, the one choosing among the options that satisfy a request; its preferences quietly fix what gets bought and what it costs. Hotel booking is a clean instance: a high-volume choice settled on a few comparable attributes, where the pick reveals...
ViSTA is introduced, a compact adapter that incorporates irregular numerical measurements into a pretrained vision-language model's chart representations and extends pretrained language models to numerical prediction and temporal questions.
Jun-Yi Gao, Yu Shi, Ping-Zhao Hu et al.· 0 citations
Large language model-based multi-agent debate (MAD) systems are being increasingly used as complex decision pipelines in distributed processes. However, their final synthesis phase still remains inadequately controlled. Even with detailed debate logs, summarizing models are prone to fabricating smoothly written debate...
Jakub Mas{\l}owski, Jaros{\l}aw A. Chudziak· 0 citations
The findings indicate that the evaluated VLMs exhibit a consistent behavioral limitation when processing typographic structures containing multiple spatial frequency layers.
Mert İncidelen, Yamen Kashkash, A. Berker et al.· 0 citations
Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.